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PREFACE 



In July 1972, the Far West Laboratory for Educational Research and 
Development initiated a project designed to develop and test procedures 
to evaluate and disseminate information on training of educational R&D 
personnel. During the Planning Phase of the project, a collection of over 
1,000 items (e.g., books, monographs, journal articles, ERIC abstracts and 
citations, other references, microfiches, and instructional materials) was 
assembled to provide an experimental file. Later, during the Preliminary 
Design Phase, approximately 213 of these items were abstracted for use In 
pilot testing alternative dissemination methods. Six deliverable by-products 
have resulted from these activities: (1) A Source Book of Educational RDD&E 
Literature , (2) A Source B ook of Educatio nal RDD&E Case Studies, (3) Catalogue 
of Educational RDD&E Instructional Materials , (4' Educational RDD&E Personnel ^ 
and Training Abstracts , (5) Recormiended Journals and Reference s, and (6) 
Reconrnended Books and Monographs . The fourth item Is the present document. 

Educational RDD&E Personnel and Training Abstracts is a by-product of 
activities undertaken to gain experience in the problems of indexing and 
abstracting relevant literature. The selection of content to be abstracted 
was based partly on previous products of the Oregon Studies in Educational 
Research, Development, Diffusion and Evaluation (RDD&E) and the AERA Task 
Force on Research and Research- Related Training . The 86 papers on RDD&E 
appearing in the Oregon compendium (Porter, 1972) had been assembled to serve 
as a basic literature source for persons preparing for, or already at work in, 
education related to RDD&E. "Collectively the articles provide a reasonably 
exhaustive picture of the diversity and consensus of thought about educational 
RDD&E that exist among professionals within these fields of endeavor." (Op. 
cit., p. iii). The Oregon compendium deals primarily with the practical 
and philosophical issues regarding the content of educational RDD&E. 

We turned to the AERA Task Force reports (Worthen and Byers, 1970; 
Brzezlnski and Smith, 1971; Smith, 1971; and Smith, Anderson and Gephart, 
1971) for literature regarding RDD&E personnel and training issues per se. 
The collection of abstracts was expanded to include items representing other 
content and formats (e.g., case studies, instructional materials, proposals, 
etc. ). 

A standard form was created which contained five blocks: (1) o code 
block (containing the Laboratory identification number and the ERIC ED 
number, if applicable), (2) the citation block, (3) a block for the index, 
(4) the abstract block, and (5) a block for notations (variable information 
such as name or address). 

The ERIC system was searched to locate ED numbers, citations, and 
descriptors. When the ERIC item appeared in Research in Education (RIE), 
the RIE abstract was used. Unfortunately, abstracts for only a small 
proportion of the items were obtainable from RIE. The remainder of the 
items were abstracted and indexed by the project staff. 
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Three prototype products were derived, in part, from this collection 
of abstracts: A Source Book of Educational RDD&E Literature (containing 
abstracts of 39 of the Oregon compendium articles) > A Source Book of 
Educational RDD&E Case Studies , and Catalogue of Educational RDD&E 
Instructional Materials , Although portions of the three prototypes are 
redundant, the entire collection of abstracts appears only in this 
document. 

We make no claims for comprehensiveness. The abstracts W-re created 
primarily as an experimental device to assist in the design and test of 
an information system. However, the abstracts do provide a selected, 
"browsable" source of significant docunents regarding educational RDD&E 
theory and practice as well as related personnel supply and demand and 
training issues. A loose-leaf, one-item- to- the-page format has been 
selected in the hope that users vill find it easier to select, rearrange, 
copy, or add to the collection (blank forms are included). 

By way of acknowledgement, Nancy McCutchan was responsible for 
supervising the following staff of abstractors and indexers: Mary 
Burlingham, Marguerite Fischer, Mary Goldthorpe, Allison Seidel, and 
Gloria Stockton. Carol Burkhart supervised the typing, coding, and 
production of this document. 



Paul 0. Hood 
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ED Q^^ 944 Rii mi nnm \ 

The Oregon Studies in Educational RDD&E. Vol . V: A Methodology for the Study of ' 
Educational RDD&E. Final Report. i 
Carl, Loring M. (Ed.) and Others • 
Monmouth, Oregon . i 

Teaching Research i 

Grant No. OEG-0-70-4977, 1972 ' 

*Case Studies, Computer Programs, Data Analysis, Data Collection, Data ~^ 
Processing, Diffusion, Educational Development/*Educational Research/*Evaluation ' 
Criteria/*Guides , Program Evaluation/*Research Methodology, Research Utilization, 
School Visitation/ i 

The primary purpose of this volume is to serve as a "how to" manual for indlviduajis 
interested in replicating or expanding the data collected by the Studies. Two 
introductory sections discuss the development of the methodology and the organization 
of the methodology description. The three subsequent chapters, which constitute ! 
the body of the document, are devoted to a presentation of the methodology. The first 
of these chapters describes the procedures involved in generating data from the I 
sites selected for study. The second chapter describes data reduction activities, I 
while the third chapter describes the procedures involved in the preparation of a i * 
profile description of the site visited. An epilogue following the three methodology, 
chapters discusses briefly the nature of the methodology as a technological contribut,ion. 
Related documents are EA 004 582-588. 
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The Oregon Studies in Educational Research, Development, Diffusion, and Evaluation, 

Vol. I: Summary Report 
Schalock, H. Del & Others 
Monmouth, Oregon 
Teaching Research 

Grant No. 0EG-0>70-4977, Final Report, 1972 



*Case Studies, Conceptual Schemes, Data Bases, Data Collection/*Diffusion/*Educationlil 
Research/*Evaluation, Literature Reviews » Manpower Development, Manpower Needs, 
Measurement Goals. Objectives, Problem Solving, Research and Development Centers, 
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This volume serves as an introduction to and a summary report of the project 
and provides a description of the context, the rationale, and the objectives of 
the Studies* The first five chapters describe the purposes to be served by the 
study, the preparations necessary to serve those purposes, the directions taken 
and the decisions made to accomplish the work, the variables selected as sources 
for data collection, and the primary vehicle used to present these data. Three of 
the next seven chapters serve as outline maps descriptive of the personnel, the 
outputs, and the work requirements found in educational RDD&E. The remaining four 
data chapters represent detailed maps, three of which describe the data in terms of 
the outputs selected as the focus of data collection, and one of which deals 
with special analyses of the data in response to a limited set of specific ques- 
tions. The final four chapters of the volume discuss the implications of the 
Oregon Studies data for (1) conceptual and methodological development, (2) 
training personnel, (3) developing training models, and (4) further exploration 
and study. Related Documents are EA 004 583-589. 
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"Basic Research-Description Vs, Definition/* in The Literature of Educational RDD&E 

Vol- II, Part One, The Oregon Studies in Educational RDD&E 
Kidd. Charles V. 
Monmouth, Oregon 
Teaching Research 

Pro ject No, 0-Q70K Grant No. OEG>0-70-49 7 7 , 1972 

♦Definitions/*Probability/*Statistics/*Federal Aid/*Scientif ic Research/*Research/ 
♦Scientists/*Adniinistrative Pol icy/*Decision Making/ 



Definitions of basic research may be categorized as investigator-centered | 

or as substance-centered. The chief characteristic of the investigator-centered . 
definition is that it is framed in terms of the motive of the investigator. A 

subgroup of this type of definition is concerned with the ^agree of freedom I 

the investigator enjoys. In the second category, that of substance-centered ' 

definitions, a distinction is made between basic and applied research in I 

terms of the scientific significance of the findings. Criteria in each of l 

the two categories, when considered literally, lead to contradictions. t 

Underlying this situation is the confusion of "basic findings" with "basic , 

research." A reconciliation can be made between the definitions when they , 
are viewed not as literal and descriptive but as statements of the probability 

of producing a basic finding. An administrator can make his decisions on ' 

the basis of probability, but such a basis is unsuitable for the collection | 
of statistics. The predisposing factors that enter into the probability are 
too many and are not of the kind that lend themselves to measurement. Even 

so, it may be better to collect and publish statistics than not to, for our ' 
nation tends to rely upon statistics in reaching judgments* It may not be 
possible to define basic research operationally, but it is possible to provide 

funds under conditions that will promote basic findings. (MF) j 



"University Basic Research," In The Literature of Educational RDDSE, Vol. II, 

Part One, The Oregon Studies in Educational RDD&E 
DuBridge, Lee A. 
Monmouth, Oregon 
Teaching Research 

Project No, 0>070K Grant N o, OEG-0-70-4977, i972 

*Federal Aid/*Scientific Research/*Prob1 ems/*Nati ona 1 Programs/*Uni vers i ti es/ 
♦Public Policy/*Social Studies/*Technology/*Science Education/ 



The case still needs to be made for basic research, and criticisms of 
it need to be answered. The case for federal support of science was first 
set forth 20 years ago by Bush. Since then, the basic case has not been 
changed: 1) research is a prime human urge, 2) basic research has had 
practical applications, 3) new knowledge is necessary to make this planet a 
better place, and 4) scholarly inquiry educates the minds of the future. 
Still, science has failed to cure all the world's ills. The question arises: 
should we not spend our money more particularly on the problems of war, 
overpopulation, urban living, and a better way of life? The answer is yes. 
Science is not the answer to all problems. The world's problems include 
problems of human understanding. With these, scientists can help as human 
beings. But in this process, pure science should not be abandoned. Granting 
this, how does one set forth to government representatives just how much 
monetary investment is justified? In considering national policy, one of 
the points to be stressed is that most of the current federal expenditures 
for university-based research in science are aimed at the support of national 
goals; among these, military technology. Of basic research funds allocated 
to universities only the 15 percent that comes from the National Science 
Foundation is solely devoted to the support of science and scientific 
education, (MP) 
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"Applied Sclpice ani Technological Progress," in The Literature of Educational RDD&E 

Vol. II, Part One, The Oregon Studies in Educational RDD&E 
Brooks, Harvey 
Monmouth, Oregon 
Teaching Res arch 

Prn^pct Nn. 0^0701 , Grant No. OEG-0-70-4977 , 1972 

♦Technological AdvancOTent/*Scientific Research/*Technology/*Scientists/*History/ 
*Universities/*Federal Aid/*Industrial Technology/*Social Influences/*Public 
Policy/*Science Laboratories/ 



Within the spectrum ranging from pure science to technology, "appliedness" 
depends upon the time scale, the specificity of the application, the researcher^s 
motives in his choices, the existing situation in technology, and the environment 
of the research. Work within one area of the spectrum may affect another area. 
Historical studies that show technology and science developing independently of 
each other are misleading when applied to the present, for the detailed inter- 
communication of science and technology is becoming closer. Technologies 
developed for scientific purposes may become useful to industry; technologies 
developed for applied purposes may later provide instruments for pure science. 
In some instances, it may be desirable to develop a field of pure science 
partly for the technology it generates. As applied research and development 
are more and more performed by people with training in basic science, applied 
research is likely to bring increasing benefits to science itself as well as 
to technology. Many important decisions in applied science depend on social 
desirability, which must be determined by a multi-dimensional Interaction 
of scientists, technologists, public servants, and the public. Scientists 
and engineers have the obligation to inform the public intelligibly, so that 
it will make intelligent decisions. When social needs are formulated in the 
right way, technological solutions may become obvious. There needs to be a 
delineation of the roles in applied research of the federal government, industry, 
and universities. In the modern era, the characteristic institution for the 
conduct of applied research is the large, multidisciplinary "mission-oriented" 
research organization. (MF) 
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in The Literature of Educational RDD&E, Vol. II, Part One, The Oregon Studies 
Carrol 1 , John B, 
Monmouth, Oregon 
Teaching Research 

Project No. 0^0701 . Grant No. 0EG^n^7n- 4977 , 1972 
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Basic research in education is needed, and it is desirable that funding 
agencies support it. To gain this support, it is useful to distinguish 
between basic and applied research. It has been said that science is basic 
to the extent that it has a high probability of yielding a new scientific 
finding; and also that it is a game, a form of play, Basic science is 
addressed to an understanding of phenomena and applied science to the 
achievement of a practical goal. In the behavioral sciences, basic research 
often has to do with a "molecular" level of behavior, while applied research 
has to do with a "molar" level of behavior. Basic research relies to a greater 
extent on models of functional relationship that involve relatively small 
error components, while applied research tends to use models that are more 
probabilistic and error-laden. More often than applied science, basic science 
is conducted in the laboratory and is concerned with theory. Basic research 
in education led to programmed instruction and is again needed in that field. 
Research on psychological processes is being done in the fields of reading, 
individual differences, and creativity. The due proportion of support that 
funding agencies should give basic research in education is estimated at 
between 15 and 25 percent of the educational research budget. (MF) 



ERIC 



Fn nn FWL 012 0008 

Criteria for Methodological Adequacy for Research on Educational Change 
Also in the Oregon Studies in Educational RDD&E, Vol, II, Part One 
Gephart, William J. 



September, 1965, 60 p. 

♦Educational Change/Educational Research Techniques/Research Methodology, Models, 
Evaluation, Research, Milwaukee/ 



Research adequacy must be assessed and standards drawn if progress is to be made 1p 
the accumulation of knowledge. This discussion of methodological criteria focuses | 
upon the following topics--(l) A logic framework for educational research, (2) 
General criteria for research evaluation, (3) Elements of the study of the Educationkl 
change process, (4) Methods and techniques for studying the change process components, 
and (5) Criteria of adequacy for evaluating research techniques in the study of ! 
educational change, I 
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"A Question Cycle to Aid the Investigator," in The Literature ot tducational Kuu&t, 

Vol. II, Part One, The Oregon Studies in Educational RDD&E 
Sheehan, T. Joseph 
Monmouth, Oregon 
Teaching Research 

Proiect No. 0-070 1 . Contra c t No . OEG-0-70- 497 7, 1972 ^ . . . 

*Educati onul Research/*Def i ni ti ons/*Eval uati on/*Experiments/*uata Ana iys 1 s/ 
*Classification/*MeasuremeRt/*Validity/*Re1iability/*Statistical Analysis/ 

♦Design/ 

To make"explicit some of the questions and issues which the investigator must 
eventually answer, this paper identifies a cycle of five steps, each of which is 
related to every other. They are: 

1. At what point along a continuum ranging from Evaluation, Data Analysis 
through Experimental Research will the work be carried out? 

2. Can a balance be struck between all the questions of interest and those 
that might reasonably be answered? j u- u 

3. In terms of design, what ways of classification are important and which 
are absolutely imperative? Can these be ordered or ranked in terms of their 

importance? . ^, • 4. o 

4. What measures will be used? Which are response variables, covariates? 
What about their validity in terms of the questions raised in No. 2? What about 
their reliability, i.e., error and "watering-down" effects on the questions raised 
in No 2? 

5. Statistical analysis will involve fitting an adequate mathematical 
model to the factors which emerge from the above questions; and, in particular, 
as the model is built, some further questions must be answered concerning the 
hypotheses, the response variables, and the covariates. (MF) 
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"Team Research in Education," in The Literature of Educational RDD&E, VoK II, r 

Part One, The Oregon Studies in Educational RDD&E t 

Allen, Dwight W. < 

Monmouth, Oregon i 

Teaching Research i 

Prnject Nn. 0>070K Grant No, 0EG-0-70-4977, 1972 » 

♦Educational Researchers, Research Committees, Educational Research/*Remedial ^ 

Instruction/*Instructional Material s/*Career Choice/*Field Studies/*Community ^ 

Cooperation/*Classroom Research/*Models/*Risk/*Educational Change/ 1 

I 

. L 

The lone researcher, dreaming in his ivory tower in a university, is not | 

doing enough for education. What is needed is an imaginative research structure, . 
using the team approach and linking research and practice. A team might be 

composed of academicians, teachers, administrators, students, outsiders from j 

the local community, and anyone else who might contribute. The team could ' 

change its composition, organization, or problems and could disappear when I 

its function was fulfilled. It should be flexible and should be committed I 

to a range of experimentation that could find and implement alternatives to | 

current failures. Team organization should allow the context in which problems ^ 

exist to dictate the directions in which solutions are sought. Team research | 
is needed that expends not only money but also such resources as student and 

teacher time and talent. Furthermore, there should be remediation for the ^ 

effects of experimental failures. A research team might be able to do a 1 

better job of developing instructional materials by securing the cooperation , 
of surrounding community schools. Research team participation could become 

a primary career goal. Educational research needs to be adventuresome; it ' 

needs to Include field experience. In education, risk should be rewarded, ' 

change institutionalized, and theory discovered through practice. Educators | 

need to discover the clear needs of the real classroom. (MP) ' 
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in The Literature of Educational RDD&E, Vol. II, Part One, The Oregon Studies 
Tyler, Ralph W. 
Monmouth, Oregon 
Teaching Research 

Proiect Nn. 0^0701, Tyrant No. 0 EG-0-70-49y7 , 19 72 

*Educationa1 Research/*Research Design/*Science Education/*Psychoeducationa1 
Processes/*Curri cul um Eval uati on/*Sci enti f i c Methodol ogy/*Def i n1 ti ons/*Mode1 s/ 
Research Cri teria/*Socid1 Class/*Perception/*Educationa1 Objecti ves/*Science 
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Much of the current research effort in science education is inadequate. Faults 
discovered in a review of abstracts of this research included: lack of generaliza- 
tion, ill-defined or gross variables, and short time span of investigation. What 
research is needs to be understood: its objective is generalization. Educational 
research should provide an understanding of the educational process and a basis 
for developing educational programs. The value of this research is that it gives 
the practitioner broader and more detailed maps of the terrain of education. 
Social psychologists have brought to these educational maps the concept of 
subjective perception. Sociologists have added the concept of social class. 
Against the background of these contributions, three kinds of criteria are 
suggested: 1) relevance, 2) adequate conceptualization, and 3) sound methodology. 
With these criteria in mind, it may be helpful to review an area in which not 
enough educational research is being conducted— the development and improvement 
of the curriculum. Research is needed to assess the understanding of teachers 
and students as to the objectives and learning experiences involved in new courses. 
Research should also be undertaken to determine the extent to which courses as 
conceived and planned are actually being carried out. Appropriate measuring 
instruments for the objectives of the new courses need to be developed. (MF) 
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"Resources ♦ Models and Theory in the Improvement of Research in Science Education," 

in The Literature of Educational RDD&E, Vol. II, Part One, The Oregon Studies 
Tyler, Ralph W. 
Monmouth, Oregon 
Teaching Research 

Project No, 0-0701, Gra nt N o, OEG-0-70-4977. 19 72 > 

*Science Education/*Educational Research/*Models/*Educational Resources/*Learning 
Processes /*Teachi ng/*Learni ng Experi ence/*Student Devel opment/*Educati onal 
Objectives/*Student Evaluation/ 



A comprehensive map of the terrain of science education needs to be constructed 
in order to improve research in this field. Areas into which the terrain might be 
broken include the following: 1) the objectives of science education, 2) the 
teaching-learning process, 3) the organization of learning experiences, 4) the 
outcomes of science education, and 5) the student's development. Research into 
objectives should provide studies of the contribution science can make to the 
human repertoire of behavior, the possibilities inherent in man, and the level 
of generality or specificity required of an objective. Research into learning 
might investigate the influences of school climate, peer groups, identification, 
and teacher effectiveness. More research is needed into the way learning experi- 
ences in science are organized: principles of sequence and integration need to 
be investigated. Research is needed to determine the persistence of independent 
learning and to develop better instruments for measuring outcomes. Investigations 
into student development in relation to science education might examine the 
decisions made by young people that result in attitudes toward science* the 
influence of the family on those attitudes, and the differential development 
of cognitive processes. Resources that are required to improve science 
education are highly competent research people, appropriate library facilities, 
places for experimental teaching and learning, opportunities for science education 
centers to communicate with each other, and university commitment to science 
education research. (MF) 
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"Elements of National Science Policy: A Perspective from the Behavioral & Social 

Sciences," in The Literature of Educational RDD&E, Vol. II, Part I, Oregon Studi 
Gideonce, Hendrik D* 
Monmouth, Oregon 
Teaching Research 

Prnject No. 0-070K Gr ant No> OEG-0-70-497 7 . 1972 



*Public Policy/*National Programs /^Federal Aid/*Social Sciences/*Behavioral 
Science Research/*Research/*Distinctive Feature/*Scientific Research/*Development/ 
*Technology/*Scientific Manpower/*Decis1on Making/*Definitions/*Management/ 



In a presentation to a Congressional subcommittee on science, research and 
development, these points were made: 

1. A serious imbalance currently exists in the policy discussions regarding 
science in this nation, an imbalance which slights the behavioral and social 
sciences. 

2. Solutions to crucial problems confronting the nation are more closely 
tied to our knowledge of individual and group behavior than to the physical 
or natural sciences. 

3. While there are similarities between our understandings of research and 
development in the natural and physical sciences as contrasted to the behavioral 
and social, there are also important differences. 

4. One of the most important differences is the political and social character 
of the behavioral and social sciences. 

5. A policy is proposed whereby the present 97:3 ratio in favor of the natural 
and physical sciences is changed by the year 2000 to 70:30. 

6. In order to accomplish this goal, policy studies will need to be done 
on the behavioral and social sciences, the structures established for making 
decisions in those sciences, and the manpower and institutions engaging in such 
work. 

7. Implementing the policy will entail different but coordinated responses 
on the part of the National Science Foundation and the mission-oriented agencies. 
Although this argument was made in terms of funds, what is needed first is 
understanding and analysis. This Nation can do what it wants to; the real 
question is one of priorities. (MF) 
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The Ten Most Significant Educational Research Findings in the Past Ten Years 

Also in the Oregon Studies in Educational RDD&E, Vol. II, Part One 
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To ascertain the significance of research in education, three questions may be j 
asked--{l) To what extent has the behavior of individuals in education been changed?| 
(2) How many articles have been written as a result of the findings? (3) To what 
extent do educators talk about or use the concepts generated by the research in j 
discussing their own problans? Application of these questions leads to identification 
of the following significant lines of research--(l) Study of quantitative developmeht 
of human characteristics, (2) Studies of the nature of human intelligence, (3) I 
Applications of computer technology to research in learning theory, (4) Investigatiqns 
of the effects of environment on the development of individuals, (5) Inquiries intc^ 
the development of the individual, (6) Research into characteristics and styles of | 
school administrators, (7) Identification of secondary school students' interests, 
career plans, and relation of school courses to their objectives, as exemplified in ' 
curriculum revision projects, (9) Conant*s studies of American schools, and (10) ( 
Determination of biblbgical bases for learni.^g and memory. This document is Volume .6, 
Number 10 of the "Executive Action Letter" for May 1967, and is available for $1.25j 
from Croft Educational Services, 100 Garfield Avenue, New London, Connecticut 06320. 
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To understand better his own teaching and to contribute more to the science \ 

of teaching, each educator should examine the full countenance of evaluation, | 

It has its formal and informal sides and appears to be changing. What it can ^ 

and should be is indicated here. The specialist in evaluation seems to be . 

increasing his emphasis on fullness of description; but most evaluation ' 

specialists have chosen not to judge. What is apt to happen is that evaluators ! 

will seek the judgments of others, such as, society at large, subject- I 

matter experts, teachers, parents, and students • The educator-evaluator can | 

organize his data-gathering to conform to the matrices shown: a description , 

matrix and a judgment matrix. In the description matrix, data may be I 
recorded under "intents" and "observations." In the judgment matrix, data 

may be entered under "standards" and "judgments." Antecedents, transactions, • 

and outcomes, the elements of evaluation statements, have a place in both I 

description and judgment. There are two principal ways of processing descriptive . 
evaluation data: finding the contingencies among antecedents, transactions, and 
outcomes and finding the congruence between intents and observations. Judgments 

may be relative or absolute. Evaluation may be formative or summative. | 

Educators should make their evaluation more fonnal in order to avoid the J 

restrictive effects of incomplete guidelines and inappropriate countenances. ^ 

(MF) , 
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The need for competent formal evaluation programs, particularly for new federally 
assisted programs, is expressed. Problems in defining educational evaluation and its 
requirements, in designing such evaluations, and three possible sources of faulty 
conceptual bases for evaluations, are presented. An attempt is made to define 
evaluation in general, to analyze emergent problems of educational change, and to 
Identify the types of decisions for which evaluations are needed in these programs. 
Four strategies for evaluating educational programs are outlined. These include 
context, input, process, and product evaluation, each of which are used at a distinct 
stage in the development of a program. Finally, a general guide for developing 
evaluation designs to implement a given evaluation program is provided. The logical 
structure of evaluation design is presented in a step by step format. 
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An adequate methodology of curriculum evaluation involves considerable 
complication of the model of evaluation study. Evaluation may have goals, 
and it may entail several post?>t;le roles* It may be formative or summative. 
Some kinds of process studies are similar to evaluation studies, but the two 
are not identical. If evaluation has any reference to goals at all, it must 
have procedures for evaluating them, 'intrinsic'* evaluation and "pay-off" 
evaluation represent two pure alternatives, with the possibility of a com- 
promise. Mediated evaluation has some advantages over pay-off evaluation, 
but the issue is not completely one-sided. Comparative evaluation is preferable 
to noncomparative evaluation. To criticisms of control group study, the 
suggestion is made that several groups may be used and compared. Good evaluation 
should not only be concerned with evaluating the student by criteria as given, 
but also, with secondary and tertiary effects. To be complete an evaluation 
should be concerned with values and costs. Some evaluation is explanatory, 
but this kind of evaluation is secondary to the kinds discussed. The expense 
of effective evaluation is high, but so is the social payoff, and our society 
can afford it. (MF) 
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There is on the scene of education a new professional--the evaiuator, Tfns is , 
a person who thinks about and plans the evaluation of educational processes. 

Because his role is a new one, it needs to be more clearly defined. Some of the ' 

assertions that have been made are that evaluators should evaluate the goals of ' 

instructional programs, that they should apply absolute standards obvious to I 

themselves, that they should be able to appraise instructional programs independent i 

of their effects on students, and that they should ignore teachers who feel I 

threatened. But if evaluators generally took such an absolutist position, a number | 

of unfortunate consequences would follow. Assumptions that provide an alternative ^ 
are that society should be served by its educational institutions, that society 
should set the goals, that an evaluation should consider the effect of a program 

on the students, and that educational goals should be stated in descriptive i 

rather than interpretive language. The function of a professional evaiuator should | 
be to help teachers and administrators to define their goals in terms of pupil 
performance, to discover differences among pupils that require particular kinds 

of instruction, and to design and administer evaluation programs. Hopefully, ' 

the research and development evaiuator will be able to bridge the gap between ' 
the laboratory and the field. (MF) 
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A symposium was conducted on the evaluation of Title I programs* The i 

contributors--James S. Coleman, David G. Hawkridge, Albert B. Chalupsky, I 

and Mi^^hael Scriven— were asked to respond to these questions: 1) What, if ^ 

any, are the differences between research and evaluation? 2) What should , 

be the focus of an analysis of a Title I program? 3) How bound to the ' 

"givens" of a program should an evaluator be? Coleman states the reason for ' 

a formal educational evaluation, the importance of determining criteria in I 

terms of customer needs as well as wishes, and the importance of examining I 

inputs as received by the child as well as inputs disbursed by the educational i 

system. Hawkridge and Chalupsky describe an ideal research and evaluation i 
model, and how it is affected when applied to the evaluation of a Title I 

program.. They recommend that the evaluator be objective, and that the same ' 

amount of money be devoted to thorough evaluations of fewer programs. I 

Scriven criticizes the evaluation reports he has examin:- for inconsistency, , 

parochialism, relativism, informal ism, and inconclusiveness . He states that ^ 
it is the evaluator's job to condense his data into one word: "good" or 

"bad." He also states that evaluations are not matters of opinion or taste ' 

but of fact and logic. CMF) J 
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This paper is a chart, listing five evaluation models and their aspects. 
The models are: 1) Ralph Tyler's Evaluation Model, 2) the School Accreditation 
Model, 3) Bob Stake's Countenance Model, 4) Dan Stufflebeam's CIPP Model, and 
5) Hilda Taba's Social Studies Evaluation Model. The aspects are these: Key 
Emphasis, Purpose, Key Activities, Key Viewpoint Used to Delimit Study, Outside 
Experts Needed, Expected Teaching Staff Involvement, Risks, and Payoff. As 
an example of the use of the chart: for Ralph Tyler's evaluation model the key 
enphasis is cn instructional objectives; the purpose is to measure student 
progress toward objectives; the key viewpoints used to delimit the study are 
those of the curriculum supervisor and teacher; the outside experts needed are 
objectives specifiers and measurement specialists; the expected teaching 
staff involvement is that of conceptualizing objectives and giving tests; 
the risks are that school aims will be oversimplified and process ignored; 
the payoff is that of ascertaining student progress. The payoff for model 
2) is that it Increases staff leadership responsibility; for model 3) the oayoff 
is a broad picture of curriculum and conflicting expectations; for model 4) 
the payoff is a curriculum sensitive to feedback; and for model 5) the payoff 
is that it supplies rules for developing new programs. (MF) 
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This Stufflebeam-Guba CIPP type model for the evaluation or innovations in educatic 
attempts to maximize the effectiveness of critical decisions through the timely reportir 
of- relevant information in a useful form to appropriate levels of decision making. ' 
Evaluation is thus seen as the combination of effective decisions based on timely, ' 
relevant Information. The system focuses on four classes of decisions and is designled 
to yield four kinds of information to serve those decision situations. These four i 
kinds of evaluation are context evaluation, design evaluation, process evaluation, j 
and product evaluation. Context evaluation consists of planning decisions and the | 
context information that serves them. It deals with the setting of priorities and the 
selection of strategies. Design evaluation entails structuring decisions which depend 
on design information. In this phase, objectives are specified and means to attain 
1 them are selected. Process evaluation deals with the possible need to restructure i 
the program after results of pilot testing and previous evaluations are in. Product 
evaluation considers evidence about the program's effectiveness in attaining its 
overall goals. Problems in applying this system might include identifying decisions 
and decision makers, timing decisions, identifying relevant information, and reporting 
information in a useful form. ^ 
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The Curriculum Evaluation Model is a technique for calculating the cost- j 

effectiveness of alternative curricular materials. Requirements that the . 
materials must meet fall into four major categories: coverage, appropriateness, 

motivational effectiveness, and cost. These categories do not lend themselves ! 

equally to objective evaluation. While the cost of curriculum materials may ' 

be represented in dollars with a high degree of accuracy, the measurement I 

of the factors in the other categories must rely on the judgment of individuals. f 

Their objectivity is ensured by the establishment of standards which the I 

ideal curriculum should meet and by the assignment of weights to each of , 

these standards. Questionnaires are used to determine the preferences of i 
students, teachers, and administrators. Processing the questionnaires precedes 

completion of the evaluation. This aspect of the model is intended to facilitate ( 
understanding in those who use the curriculum materials; for only when the materialsj 
are accepted can they be effective. The model suggests guidelines for a 
quantitative measurement of the curriculum. It should be noted, however, 

that a high aggregate score is not invariably the sign of highly desirable ' 

curriculum materials. The user of the model is cautioned to adhere to the ' 

v;eightings suggested by it in his first evaluation, even though the same ' 

numerical weighting may not be equally applicable to all situations in alT ' 
schools. An outline of components and weights, a rating form, and questionnaires 
for students and teachers are appended. (MF) 
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Even a public school researcher who has not been involved in the planning \ 

or execution of a project may provide sufficient evaluation of it for the 1 
decision-makers to be able to make a decision relative to it. An example of 

this is cited. A research group was asked to evaluate a Community School j 

Advisory Committee in one of the transitional areas of Philadelphia. The ' 

first action of the researchers was to review the Board Resolution which I 

instituted the Committee. They found that there was a statement of objectives, I 

but that the objectives were loosely worded. The researchers then set out I 

to determine the objectives of the project. This they did by an assessment , 

of the perceptions of the people involved, based on their spoken and written | 
words. Then an analysis was made of the actual behaviors of the people in 

the project. Criteria by which the Committee was to be evaluated was then I 

established. It was determined that the Committee had accomplished most of | 

the goals that had been set. But that it had not been effective in meeting : 

its goal of communicating with the community. On the basis of analyses ' 
utilizing only post hoc research, the researchers were able to give the 

Board of Education the information it needed in order to make its decision ' 

about the Committee's continued existence, The project was continued, and the | 

research team was asked to participate in it throughout the next year. (MF) ^ 
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A model based on value judgments was developed to provide information to 
be used in funding decisions. It was developed as one i^hase of an 
evaluation of an ESEA Title I program in the School District of Philadelphia. 
The 35 projects already in operation within the ESEA program were analyzed 
to discover the basis of previous funding decisions. This basis was found to 
consist of 20 kinds of key decisions grouped within four major classifications 
of service: services to pupils, to staff, to community, and to the system. 
A matrix was then developed to be used for individual project assessment. 
The outcomes of such assessment produced a rank ordering of projects based on 
school district priorities. To get a ranking of dollar-weighted values per 
project, this formula was then used: Value=Judgment Score/Weighted Pupil 
Cost. When funding decisions were made for Title I projects for the current 
year, selection of new projects was screened using the guidelines in the 
judgmental matrix. Funding decisions, however, did not entirely correspond 
to the priorities identified by the matrix. It is concluded that use of 
this model seems to be most appropriate during the program planning operation. 
The model offers a systematic tool for decision-makers who are faced with 
deciding the relative merits of project proposals. (MP) 
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When and how can an educational program be evaluated so that the program i 
can give the poor a better education? The answer is that evaluation must supply the, 
administrator with pertinent and timely information at each decision step in 
program development. To provide this information, the Discrepancy Evaluation ! 
Model was devised. It is to be understood as operating under stated limits and ' 
conditions and as proceeding in five stages. The first stage of the model is ■ 
the design, consisting of three sections called inputs, processes, and outputs. i 
The second stage of the model provides information on whether all the input i 
design elements are present and whether activities and operations are being i 
carried out as described in the process section. Measurement begins with Stage | 
III: the interim products of each of the process elements are measured. Any 
discrepancy found between the performance and the standard is reported back to ' 
the program staff. In the fourth stage the question is asked whether the I 
program has achieved its goals. In Stage V a cost-benefit analysis is under- : 
taken. The model has been applied to a poverty school program called the ' 
Transition Room and, after several years, is now at Stage III. Two new programs 
have been initiated and are being aided by the use of the model. (MF) ' 
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An approach to evaluating an educational program undertaken to benefit 
children is to begin with a study of the children entering the program. A 
base line should be established, recording each child's previous experiences, 
the vicissitudes to which he has been subject, and some assessment of his 
family, home life, neighborhood, and housing. Also, at the beginning of the 
program, a comprehensive set of measurements should be taken of the child. 
These should include his height, weight, various physiological functions, and 
motor capacities. His nutritional condition should be noted. Measurements 
and tests should be made of his various sensory acuities, his nervous system, 
and his cognitive capacities and performances* Efforts must be made to observe 
and record his personality, emotional reactivity, jgroup behavior, and self- 
discipline. His image of himself should be discovered as far as possible. It 
would be helpful to calculate the individual child's percentile rank order in 
his group. To convey the multi-dimensionalities of each child, the percentile 
ranks he exhibits may be graphically presented on polar coordinates. By 
scanning these graphic devices, it will be possible to visualize the status and 
conditions, the strengths and weaknesses in the context of the whole child, in 
so far as he has been observed and recorded. It will then be possible to 
evaluate the effect of a program on individual children. Group success or 
failure will not be the sole criterion of the success or failure of a 
program. (MF) 
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In tTie evaluation of newly developed curriculum materials, two kinds of \ 

investigations should be made: 1) into the effectiveness of the new materials . 
in getting students to learn the subject matter, and 2) into the effect of 

the materials on the students' general perceptions of the subject and its I 

modes of inquiry. Such investigations were made into the effectiveness and ' 

effects of Charting the Universe , Book 1, prepared by the University of ! 

Illinois Elementary-School Science Project (ESSP). The population used in t 

the study consisted of 92 students, 43 boys and 49 girls, in the fifth grade | 

at the University of Chicago Laboratory Schooi. The students were taught by | 

the same teacher in four instructional groups of 23 students each. The range of j 
I.Q. scores for the entire group was 88-179, with a median score of 124. Pre- 

and posttest data from specially constructed subject-matter achievement tests ' 

and I.Q. scores from the Henmon-Nelson Test were analyzed to test the hypotheses ( 
concerning effectiveness. Assessment of the effects on students of studying 
the ESSP materials was made with the help of the Test on Understanding Science 

(TOUS), Form Ex and a semantic differential instrument, called Word Association ' 
Stud y (WAS). The effectiveness of the materials was found to be moderate; the 
effect on the students, varied, but generally slight. More important, perhaps, 

than the findings are the procedures of analysis and interpretation that were * 
used. (MF) 
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Formative evaluation was undertaken of a kindergarten unit Curves and Shapes , 
with seeming improvement in the revised unit. The evaluation proceeded in 
accordance with the Domain Referenced Achievement Test Systems (DRATS), developed 
by the Minnesota Mathematics and Science Center (MINNEMAST) Staff. The purpose 
of DRATS was to provide authors with information on student achievement related 
to unit objectives, so that the authors might make decisions about the units. 
In the first set of tasks in the evaluation, the unit was reviewed by the 
evaluator who then deduced a set of prototype items (suggested student tasks). 
The next step in the development of item forms for a unit was an encounter with 
the authors. When a preliminary set of item forms had been constructed, a second 
encounter with authors was necessary. The result of these endeavors was a set 
of item forms felt to be the major objectives of the unit or to represent an 
interesting research question. In all, there were 13 item forms and 1032 items. 
After testing students on Curves and Shapes , a general meeting was held with the 
authors to discuss the results of the testing. A decision was made to revise 
Curves and Shapes . After this revision, a revision was also made to revise 
forms. A second cycle of testing students was undertaken, but due to technical 
difficulties, was not completed. The authors of the unit expressed the belief 
that improvement had occurred. (MF) 
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In an attempt to improve Head Start evaluations, several methodological techniques 
are proposed. Since programs vary in approach, evaluations must be made on the sucqess 
of the individual programs. Formulation of research questioFis should provide infor;ma- 
tion as to the process and outcome of the program. To avoid experimenter bias, i 
experimenters should be selected on the basis of their disengagement from Head 
Start. A baseline group (either a control group or the experimental groups assessed 
on pretreatment performance) should be used, and variables affecting their behavior' 
should be noted. Some of the problems due to the lack of measuring instruments I 
could be avoided if experimenters would not measure specific behavior as indicative i 
of general ability. To avoid the problem of publishing only positive Head Start i 
reports, the Office of Economic Opportunity should publish annually all Head Start 
evaluations. Several references are included. 
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Curriculum evaluation is affected by the context of the curriculum project 
and by the politics, ethics, and methodology of evaluation. The curriculum 
project is a new process for preparing classroom materials and for speeding 
up the innovative process in curriculum. Characteristically, it consists of 
teams of writers (usually school and college personnel) brought together for 
a relatively short time to prepare experiinentcil editions of new materials. 
To evaluate the work of a curriculum project, it is necessary to have a clear 
idea of what curriculum is. It is more than a textbook or the sum of instructional 
materials. Many curriculum developers have to come to believe that curriculum 
encompasses all school-learning experiences of the child, including unplanned 
ones. Curriculum developers have also learned that curriculum revision is never 
finished; this affects the scope and duration of the evaluative function. Another 
factor affecting this function is the extent to which the project can afford the 
consequences of systematic general evaluation; not all funding sources recognize 
the uncertain nature of innovative experiments. Evaluative information is often 
obtained by an invasion of privacy, a factor the evaluator must consider. Yet 
another fact of life that the evaluator must be aware of is that the evaluation 
must fit into the project, not the project into the evaluation. His methodology 
must adjust to this. (MF) 
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Advances and accomplishments in curriculum evaluation are examined here i 

under four headings: State of the Art, Evaluation in Curriculum Development i 

Groups, Evaluation Strategies, and Techniques and Technology. Particular , 

attention is given to the evaluation of science education. With a few notable i 
exceptions, the curriculum movement in science, characterized by its enlistment 

of scholars and scientists In federally funded projects, has not been subjected ' 

to rigorous research and evaluation. With Office of Education support, however, ' 

a change is taking place. Perhaps, the lack of examples of successful programs i 

has motivated the considerable evaluation of evaluation that has occurred in i 

recent years. This work has resulted in a number of evaluation paradigms. i 
Results of the implementation of these strategies are scarce, but they have provoked 
considerable interest. Instruments have been developed to test achievement, student 
performance, changes in attitudes or interests, critical thinking, the social climatel 

of learning, and process skills. Although most published studies involved some type , 

of experimental design, the designs tend to be weak. The conclusion affixed to , 
this review of the literature is that concentrated group research needs to be 
encouraged and supported. (mF) 
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The design for context evaluation may be described through a series of steps 
that involve: focusing the evaluation and collecting, organizing, analyzing, and 
reporting the information. In focusing the evaluation, steps should be taken 
to identify the components of the instructional program, the levels of decision- 
making, the objectives for each of the components, and the factors to be considered 
in the collection of information. Before gathering information, the source for 
the information should be identified, the possible sampling procedures should be 
examined, instruments for data collection should be selected or developed, and a 
master schedule should be set forth. Information should be organized by an 
information processing and storage system. This system should provide for the 
processing of data, a research bank, an innovative practices bank, and a central 
information storage system. Once data regarding the program have been generated, 
a preliminary report is prepared, changes based on need and opportunity are 
recommended, and a final report is prepared for dissemination. The system for 
context evaluation as described here is the first step in combining two models 
for evaluation: the EPIC Model and the CIPP Model. (MF) 
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An evaluation framework that is common to each of 47 different public school 
projects is this question: How many..*of what types of children. . .received how 
much..,of what kinds of service. . .under what conditions or through what processes 
...from how many. ♦.of what types of personnel .. .with what outcomes .. .at what cost? 
In attempting to answer this question, the evaluator encounters a number of key 
methodological issues. The specific issues considered here are: 

1. The issue of randomized experimental and control groups (in planning 
the basic design); 

2. The issues of multiple variables and of collecting data from a sample 
of participants vs. all participants (in planning data collect on); 

3. The issue of institutional expectations vs. research realities (in 
planning data analysis and reporting procedures). 

In issue 1 we are considering the elements that influence the possibility of 
obtaining randomized experimental and control groups. Very often, these elements 
influence the possibility negatively. In issue 2, the number of proposed 
variables must be limited by available time, money, and manpower. Variables 
must also be limited sometimes by the sensitivity of the data. As to the 
issue of sampling: it is often preferable in a school setting to collect the 
data from all participants. In issue 3, timing poses a major problem in 
meeting institutional expectations. Report-writing style poses another problem 
that must be solved* A humorous illustration is appended. (MF) 
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Educational development appears to be a vast and promising enterprise, even 
though reliable ways of effecting educational improvement has yet to be identified 
either by researchers, manufacturers, or school personnel. Formative evaluation, 
which has as its purpose the improvement of instructional programs, has been 
bogged down in a content-oriented approach. Its procedures would become more 
meaningful, if its approach would become product-oriented. A product is defined 
here as consisting of the materials and accompanying procedures to accomplish 
specified instructional outcomes. Three criteria can be applied to a product; 
reliability, utility, and cost* When these three criteria are met, it is 
reasonable to talk about cost efficiency and cost effectiveness. Formal public 
education at present can be characterized as print-mediated, group-paced, managed 
against relative norms, and teacher-based. Two approaches can be used in the 
context of this system. One approach involves a series of straightforward 
trial-revision cycles to sharpen the accomplishment of given instructional outcomes; 
the other involves a major manipulation of a system dimension, producing a new 
generation of the product. Each of these two approaches involves convergent 
iterative methodology. A sequence found useful with the first approach is: 
1) preparation of instructional specifications, 2) component preparation and 
tryout, and 3) product preparation and tryout. The second approach Involves 
complex development work. Associated with educational development is challenge 
of "produce or perish." (MF) 
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Multifaceted efforts to solve social problems offer advantages, but pose i 

problems for the evaluator. One such effort has been under evaluation for the . 
past 18 months. A multi-agency approach to the economic, educational, and social 

problems of rural America, it is known as Concerted Services in Training and I 

Education (CSTE). There are four answers to the question, why evaluate? They ' 

are: ! 

1. An evaluation may provide answers to pertinent questions. I 

2, An evaluation may be used to legitimize a program, I 

3, Evaluative activities may have salutary effects upon the action program , 
itself. j 

4. An evaluation may contribute basic information applicable to related 
subject areas. * 

Seven tasks are Indicated in the overall research design of program evaluation, ( 
These tasks and comments about them are as follows: 

!• Identifying the research problem--usually identified when the client ' 

approaches the researcher. ' 

2. Selecting or developing an evaluation model--the CIPP model has been ' 
proposed recently. ' 

3. Operationalizing objectives, identifying their assumptions, and determining J 
their priorities— it is helpful to analyze the more general and far reaching 
objectives into steps or subgoals. 

4. Formulating strategies and devising research techniques. 

5. Collecting the data--may be done in terms of context, input, process, ' 
and product. ' 

6. Analyzing the data--elaborate statistical techniques would not be i 
feasible for data obtained in many surveys. i 

7. Publishing the results— reports of an evaluation may be directed toward i 
program administrators, newspapers, and scholarly journals. (MF) i 
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Qualitative evaluation of formal educational courses offered through stations 
in British Columbia and the State of Washington revealed merits and faults in 
presentation and resulted in recommendations for improvement. The investigators 
were 24 graduate students from the University of British Columbia. The instrument 
they used consisted of two parts: "Objective Evaluation" and "Subjective 
Evaluation." The first part was concerned with program design and administration; 
the second part, with the instructor, studio techniques, ideas, and administration 
of the program. The findings were that course objectives were not given in the 
programs; organization varied from good to poor; the lecture was the teaching 
technique most widely used; studio techniques were outstanding; promotion of the 
programs was limited; time schedules were inconvenient; and evaluation devices 
were not used to measure the achievements of the viewer. Recommended changes are 
that the educator use the expertise of the media specialist; that course objectives 
be stated in behavioral terms; that the viewer be made an active participant; that 
syllabi and viewer's guides be provided; that communication between the instructor 
and the viewer be established; that questionnaires or interview instruments be used 
to determine whether course objectives have been met; and that the programs receive 
better timing and promotion. There is a burgeoning public interest in adult 
education that television should capture and that it has not, as yet. Continuing 
qualitative evaluation is needed to meet this problem, (MF) 
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Four models illustrate a systems approach to evaluation. Adapted from the | 
CIPP Evaluation Modal (Ohio State University), they are in the areas of: 
context evaluation, input evaluation, process evaluation, and product evaluation. 

Context evaluation examines the ecology or environment in which a program is J 
started or is currently operating. Input evaluation examines student characteristics', 

public relations techniques, admission and course selection policies, teacher \ 

recruitment and characteristics, etc. Process evaluation identifies critical I 

elanents in an on-going program over a period of time. These elements are I 

administrative policies, in-service training, teacher supervision, allocation , 

of funds for instruction, use of advisory committees, pupil grouping, etc, , 
Product evaluation identifies program success criterion measures, such as, student 

proficiencies, program-holding power, rapidity and nature of initial placement, ^ 

and student performance ratings of employers. In each evaluation model, objective, | 
method, and application are related to theory and operation. As the models are 

explained, we are to pretend that we are involved in evaluating a state system * 

of brand new vocational programs which started in 1965 and which, already, at ^ 

the close of 1968, are showing variations in program quality and effectiveness. ' 

We are to remember that each model can be used separately, if only a single aspect ' 

or cluster is to be evaluated. (MF) | 
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Educational measurement specialists should add to their repertoire of i 

methodologies. As, in the past, they have borrowed techniques and conceptual I 

schemes from psychology, agronomy, and biology, now they should borrow from , 

sociology, anthropology, economics, and history. If we consider the techniques \ 
of the past the only "right" methods, we may alter the nature of the problems 

on which we work. Very often, these problems do not accommodate themselves ' 

to a narrow class of research techniques. Scientists may define their own ' 

problems; educational researchers have their problems pressed upon them. I 

Although we have borrowed sampling techniques and survey methods from sociology, i 

we cannot brag about our use or care of these techniques. Neither can we claim i 
that we have thought much of the cross-cultural survey and field methods used 
in cultural anthropology. Nor have we utilized the different points of view 

and special methods of the economist. We have not added to our expertise, as I 

we should have by acquiring the techniques of the historian in handling "log" j 

data for purposes of inferences about change or admissibility of evidence. , 
We should see to it that all of these techniques are brought to bear on our 

problems in a concerted fashion. (MF) | 
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The hypothesis is undertaken that the role of comparative method in the 
historical developnient of branches of the social sciences is analogous to its 
role in the individual process of cognition. The first phase of scientific 
explanation in an individual or in a discipline is concerned with classification; 
the second phase, with the enunciation of causal relationships; and the third 
phase, with the confrontation of theoretical statement with empirical observation. 
Following this sequence, scientific inquiry into the educational process, coming 
in the nineteenth century, found in the comparative approach its most important 
research method. Comparisons were used then to list and classify the peculiari- 
ties, similarities, and differences of various national school systems. During 
the thirties, sociologists and economists began to deal with educational problems 
on an abstract, non-comparative level. We are now having a revival of the com- 
parative method under a new label. Correlations Approach. In this revival, 
comparisons are used for identification rather than for classification. This 
leads to the corollary that the very evolution of a field requires of comparisons 
different properties and performances in different stages of development. In 
the other social sciences--economics, sociology, and social psychology--similar 
historical developments can be demonstrated. (MP) 
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Many of the problems that face the evaluation of education programs have i 

their counterparts in business. Five such problems are: I 

1. Who should do the evaluations? i 

2. What aspects of program goals should be evaluated? 1 

3. How should program goals be set? 

4. How can programs be designed to provide clearer answers to questions 1 
regarding successful education techniques? I 

5. What about money? I 
The answers to these questions are: i 

1. To insure objectivity, evaluations should be designed, conducted, and i 
reported by an agency that has no stake in the outcome of the findings. 

2. The objectives of education programs are policy issues and should not ^ 
be evaluated by the researcher. ( 

3. Goals should be conceptualized with the help of the evaluator, so that : 
they are of a nature that can be evaluated. ^ 

4. Four of the ways to pinpoint the reasons for the failure or success of 

a program are: experimental design, magnification, asking the right questions, ' 

and segmentation. | 

5. The worth in dollars of measurable gains is a matter of policy and i 
cannot be determined by research. In all of these problem areas, industry and ^ 
marketing research have approaches that may be useful to those engaged in , 
evaluating education programs. (MF) , 
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Educational development Is a systematic process of creating new alternatives that 
contribute to the improvement of educational practice. Sources of educational 
development may be the school itself, colleges and universities, or the instructional 
materials industry. There are two types of educational development: product 
development and change support. In product development improvement in educational 
practice stems from the creation of tools, things or devices, which when used as 
directed, are known to yield desirable and specified outcomes. The change support 
approach emphasizes direct intervention, seeking to improve the behavior of those 
who are engaged in the practice of education. Most educational development is 
a mixture of these two approaches. The strategy chosen to implement the educational 
improvement depends upon the relative degree of emphasis on these two approaches. 
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Since the state-of-the-art in educational development is new, fluid, and rapidly 
growing, this paper only indicates the bounds of the domain, categorizes a few 
distinctions, and lists obstacles to progress. Development lies in the gap between 
scientific knowledge and user practices. Many think it superfluous, feeling that 
research is all that is necessary for educational improvement. Although research 
and development are related, they are separable endeavors. Pertinent distinctions 
can be arrayed as follows: 

Research Development 
Base Science Technology 

Outcome Knowledge Techniques 

Artifact Reports Products 

Time Referent Insensitive Sensitive 

Boundary Limits Variables System 

Producer Individual Group 

Control Peers Users 

Management Loose Tight 

Development efforts may be differentiated in terms of several dimensions. 
Bipolar strategy dimensions are shown below: 

Outside Education In Educatio n 

Taxonomy Management oriented Practitioner oriented 

Methodology Product oriented Change oriented 

Motivation Technology--push Requirements--pun 

Initiation Design Eolithism 

Resource Allocation Staged Linear 
Management Program Project 

The speed with which the potential of educational development will be realized 
is dependent upon a number of constraints, such as, technology, personnel, finance, 
and evaluation. Given the present state-of-the-art in educational development, 
evaluation should be aimed at nurturing growth, not eliminating competition. A 
reflection on an early NASA effort is relevant: "'A chief reason that Echo succeeded 
was that the wrong people did not realize that it was important until it was too 
late for them to interfere with it." (MF) 
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Six stages of the development process are outlined in this model. They 
are: context analysis, conceptual design, product design, pilot test, field 
test, and preparation for installation. Each stage has a process description 
that outlines the steps to be taken in development. For each of these steps 
there are criteria. And for each stage there is an outcome. The implication 
of development according to this model is that cycles will be made within 
each stage until criteria have been satisfied . As a program proceeds, it is 
not essential that each stage be completed before the next stage. Rather, 
cycles back to a prior stage may be necessary. However, changes in an early 
stage usually imply related changes in subsequent stages. The design of a 
product might be used as a one-week training program for teachers or a 
one-year curriculum for students. Within a given program, products need to 
be defined in terms of their scope. The major characteristic of a product is 
that it is replicable. A flow chart is attached to this document. It shows 
the flow between the six stages of development in the domain of research and 
evaluation. (MF) 
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Numerous groups, such as, publishers, national curriculum groups, and profes- \ 
sional curriculum groups that develop instructional materials never test them. 
Indeed, the techniques for testing them have been clearly delineated. To correct 

this situation, the activities of the developmental and evaluation process are ! 

portrayed and explained in this model used by the Rocky Mountain Educational ' 

Laboratory (RMEL). The activities are: 1) establishing educational need and ! 

problem identification; 2) specifying and delineating the problem; 3) conceptualize i 

ing a strategy to achieve the solution set; 4) implementing various types of | 

strategies; 5) developing an operational plan; 6) developing processes, procedures, , 

and materials; 7) developing an evaluation design; 8) reality and field testing; | 
9) analysis, interpretation, and recommendation; and 10) dissemination. Many 

of these processes are not linear as indicated by the nature and format of this ' 

paper but are cyclical. In other words, a potential product can move through | 
various stages of development and be rercycled back through those same stages 
until such time as the product pleases the staff. Products will probably 

seldom, if ever, be ready for dissemination without some redevelopment or ' 

redesigning. Developmental activity by its very nature sponsors more unsuccess- i 

ful than successful ventures. This is not unlike the history of developing in ' 

the fields of agriculture or medicine; but because of the low tolerance in the ' 

educational enterprise for lack of success, it seems important that it be ' 
mentioned. (MF) 
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This paper presents a model of the curriculum development process that reflects 
curriculum project practices. Regularities found in the data from one case study 
vare checked against reports of other project practices, and those regularities 
common to all the projects studied served as the model foundation. The model 
asserts that a curriculum project functions mainly to transform an initially 
vague, unsystematic, but strongly held vision of the educationally desirable 
into a concrete educational program. This transformation is acccwnplished first 
by attaining agreement on a platform—a body of shared beliefs about curriculum. 
Then, using this platform, the project staff develops a plan of work, the completion 
of which requires discussion, debate, argument, or deliberation on crucial issues, 
and finally, the production of curriculum materials. 
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A systems approach to the development of teacher education programs would improve 
teacher performance in instructional systems. An understanding of the teacher*s role 
is essential to this development. Studies ir teacher effectiveness are needed that| 
are oriented toward analysis, synthesis, and modeling of the teacher's role. 1 
Inventors of new systems must study the human components that go into the system. ' 
They must develop job descriptions as a basis for training their components. ^ 
They must create instructional objectives relevant to the teacher's role. Enabling' 
objectives will need to be sequenced and integrated to achieve terminal objectives.! 
A sequence that is suggested for teacher education emphasizes the practice of ^ 
performance. New roles may result in the future from the needs of instructional i 
systems and from the impact of educational technology. The teacher's role may 
be one of system designer, system manager, or human component. He may have ' 
multiple roles. He may be expected to deal with motivational and social influence | 
processes in interpersonal interactions with students. Descriptions are given 
of the seven steps in the development of an instructional system. The advantages 
won by taking these steps are relevancy, efficiency, and quality control. (MF) * 
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The time has come now to recognize and legitimate development as a vital form 
of inquiry in professional schools of education. To the tripartite definition 
of responsibilities in universities--research, teaching, and service--the fourth 
function of development should be added. This insistence on development in 
professional schools of education will not result in an immediate outpouring 
of development activity. Graduate training programs will continue to emphasize 
techniques and strategies more appropriate to the world of the researcher than 
the world of the developer. As a matter of fact, the methodology of development 
is still tentative. Even so, the die is cast. Social pressures are being 
exerted on the professional schools of education, and they are responding. Even 
the more reluctant participants will find themselves caught up in development 
through the necessity of training specialized personnel for other institutional 
settings. The gap between research and practice that has estranged the researcher 
and the practitioner will be filled by a new middleman--the developer. The 
developer's product is an operating system which can be deployed to carry out 
social functions. As the professional schools take up the function of development, 
the output of new knowledge and techniques will increase, and the payoff for the 
schools of this nation will be immense. (MF) 



4^ 



FWL 012 nn48 

"An Exploratory Study of Knowledge Utilization," in The Literature of Educational 

RDD&E, Vol. II, Part Two, The Oregon Studies in Educational RDD&E 
Havelock, Ronald G. and Benne, Kenneth D. 
Monmouth, Oregon 
Teaching Research 

Prn jprt No 0^0701 . Hrant. Nn. nFfi-n^7n-/lQ 77 , IQ?? 

*Research/*Comniuni cation (Thought Transfer)/*Infonnation Systems/*Feedback/ 
♦Information Theory/*Technology/*Sys terns Approach/*Administration/*Motivation/ 
*Status/*Values/*Research Utilization/*Telephone Communication Systems/ 
*Interpersonal Relationship/*Group Status/ 



Research utilization may be conceptualized both as a system and as a process. 
As a system, utilization has a "flow structure." To understand the flow structure 
of utilization two concepts are needed: these are '^barrier" and "unit of 
information." A "barrier" indicates the frame of reference of the sender and 
of the receiver. "Unit of information" refers to the substance of knowledge 
which is being transmitted. In a simple utilization system a "consumer" 
expresses a need to a "resource person" who in turn transmits the needed 
resources back to the "consumer." An example of a system transmitting basic 
research to applied research is that dfvthe American Telephone and Telegraph 
Company. Supporting utilization is administration. This support is in five 
areas: a) education, b) finances, c) control, d) protection, and e) change. 
To understand utilization as a process it may be thought of as similar to the 
processes of communication and economic exchange. Major emphasis must be 
given to the motivational aspect of utilization. Other features of process are 
"interpersonal and group membership issues" and "technical issues-" Creating 
problems for utilization are status differences and value differences. The 
technical aspect of the utilization process has two phases: the preparation 
of the message and its transmission. The schema of utilization as outlined here 
may be useful in building a program on utilization. (MF) 
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In the educational system the linker's role is an emerging one, facilitating 
the flow of knowledge between source and application. Until recently the linker 
was an individual, but now linking institutions are beginning to assume his 
role. Institutionalizing the role of linker helps to overcome some of the 
problems in the role and offers economic security to the individual. Also, a 
linking institution permits the coordination of the multiple functions of 
input, throughput, and output. Functioning much like a second linker in the 
flow of information system is the gatekeeper who links the linker and the client 
system. It is important for linkers to identify and seek out the audience 
which needs and will make use of his services. He must realize the necessity 
for all adopters to pass through the stages of awareness, interest, evaluation, 
and trial before adopting a new idea. It is difficult to bring adoption of 
new ideas to the American school system because of the scarcity of educational 
change agents, a weak knowledge base, and the domestication of the public 
schools. With an understanding of the educational knowledge flow system, the 
adoption process, and the linker's role, present performance can be evaluated 
and potential contribution assessed. (MP) 
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The major theoretical and empirical studies of knowledge dissemination and 
utilization (D&U) can be grouped into three general categories: 1) the 
"Research, Development, and Diffusion" (RD&D) Perspective, 2) the "Social 
Interaction" (S-l) Perspective, and 3) the "Problem-Solver" (P-S) Perspective. 
The RD&E model assumes rational sequence, planning, division of labor, defined 
audience, and high Investment for maximum pay-off. Major points that can be 
derived from the S-I tradition are: the importance of the social relations 
network, the user's position in that network, the significance of informal 
personal relationships, the importance of reference group identifications, the 
essential irrelevance of the size of the adopting unit, and the significance 
of stages of adoption for D&U strategies, P-S theorists stress these points: 
the user-initiated change is the strongest. Factors which account for most D&U 
phenomena are linkage, structure, openness, capacity, reward, proximity, and 
synergy. These factors chart the clearer areas of D&U; there are other areas 
that are more troubled and confusing. From the perspectives given here, 
guidelines can be derived for policy makers as well as for researchers and 
practitioners of D&U. (MF) 
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Contemporary thinking about educational innovation is characterized by several i 

generalizations. These generalizations are: 1) that educational innovation arises | 

primarily from competition and imitation; 2) that educational innovation proceeds , 

from the outside in and from the top down; 3) educational innovation involves the | 
alienation of the innovator, who acts as a "true believer;" and 4) educational 

innovation depends on the alignment of forces largely external to the school. ' 

These statements describe the real and usual situation when no intelligence is ' 

at work. Yet some of the largest organizations have shown that they can change I 

from the inside and some school systems have shown that innovation can take i 

place from the inside* The point has been made that the strategy of innovation i 
has to be comprehensive; that is, it must be carefully designed, include 
provision for local awareness and interest, provide for local evaluation of 

the proposed innovation, and provide the conditions necessary for extensive I 

local trial and adoption. A good example of a comprehensive strategy of change j 

is that of the Physical Sciences Study Committee (PSSC) Physics program. Although , 
a strategy of change is useful s the questions then arise: where are new ideas to 

be generated? and what new ideas are to be generated? Given the hierarchical \ 

nature of the school system, the success of innovation depends upon the attitudes , 

and skills--the leadership"-of the chief administrator. (MP) , 
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How can scientific knowledge be used to contribute to planned change in 
education? To answer this question its terms need to be defined: "education" 
is defined as creating and maintaining good learning experiences for children; 
"scientific knowledge" includes theory, research findings, and research methodology 
"planned change" is defined as the inclusion of certain basic problem-solving 
phases in adopting to an action concern; "utilization" of research is defined as 
both a process (a flow of knowledge) and as a structure (an organization of 
roles). These definitions can be combined into a model for educational change 
representing the process of utilization, the core of which is the problem-solving 
phase of planned change. Since research findings seldom provide the educator 
with direct answers to problems, he needs to derive implications from the 
findings. The practitioner can also borrow the methodologies of science and 
adapt these as aids in gaining knowledge of the educational setting. Scientific 
methodology may also be useful in providing the educator with evaluative feedback. 
Although it is possible that the educator can retrieve knowledge and carry out 
the phases of planned change alone, it would be more economic and valuable for 
him to use social scientists to help with some aspects of the utilization process. 
In this collaboration social scientists might gain a better perspective of actual 
situations* (MF) 
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Characteristics of the change process are described and the importance of improving 
understanding of the process of change is indicated. A summary of some relevant 
literature on the diffusion of innovation draws from studies. in rural sociology, 
cultural anthropology, industry, education, and psychology. In addition, two 
premodels of change are outlined in an attempt to conceptualize or schematize the 
process. The premodels or paradigms are intended to have some utility to practi- 
tioners and to suggest tc scholars the large gaps in knowledge that must be filled 
before a theory of change can be formulated. 
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The Educational Systems for the Seventies (ES '70) project may be viewed as a i 
linking agency, the function of which is to develop the interactive ties among the | 
many elements of the educational process in order for them to be dealt with as a ^ 
cohesive system. ES '70 may be described most appropriately in terms of its effects; 
these seem to be in the direction of self-renewal. Underlying the changes in the 
schools associated with the ES 70 project are three factors: a changed view of thd 
educational process, the provision of a local ES '70 coordinator and staff, and the I 
presence of the objectives against which a teacher can measure his contribution. I 
In the ES '70 project a multidiscipl ined organization was assigned the function of i 
linkage. The organization established formal and informal channels for communicatiqn 
for the U. S. Office of Education, the network governing board, and the local 
district school personnel, making possible two-way professional interchange. ' 
One of the significant aspects of the ES '70 demonstration was its use of separate | 
process management assistance to build and maintain the links which give a system i^s 
structure and life. Implied in ES '70 developments may be clues to the replicabili^y 
of educational demonstrations. Significantly, this does not include the need for 
massive infusion of outside funds. (MF) ' 



FWL 012 0055 ^ 

"Content and Strategies of Communication in Current Educational Change Efforts in 

The Literature of Educational RDD&E, Vol. II, Part Two, The Oregon Studies 
Horvat, John J. 
Monmouth, Oregon 
Teaching Research 

Project No. 0-0701. Grant No . OE G-0-70-4977, 1972 

*Conceptual Schemes/*Educational Change/*Communi cation (Thought trans fer)/*Written 
Language/*Meetings/*Conferences/*Demonstrations (Educational )/*Partici pant 
Involvement/*Pilot Projects/*Change Agents/*Models/ 



In a two-dimensional schema the content and strategy of educational change 
communications are indicated. Categories given for content, the "whats" and 
"hows" of change, are these: 1) communications on change efforts which have been 
or are being implemented; 2) communications about and suggestions for the use 
of change products; 3) general change ideas, suggestions, and action program 
communications; 4) communications of strategies or methods to be used in bringing 
about change; and 5) communications of theories and models of the change process 
and of the educational situation. These five content categories provide a 
general overview of the kinds of things that are being cormiunicated in the area 
of educational change. Moving to the second variable under consideration, the 
strategies for communication"-l ) there is the written word; 2) there are formal 
meetings and conferences; 3) there are demonstration, lighthouse project, and 
pilot project strategies; 4) there is the training strategy; and 5} there is the 
involvement and intervention strategy. The situation as it is cannot be viewed 
with optimism. It could be improved by conwnuni eating more of the "hows" of 
educational change, by change experts producing better products, by the use of 
more than the written word to communicate, and by offering recipients viable 
professional alternatives. (MF) 
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Planned change in the field of science education to be successful requires j 
an appropriate change strategy and a massive effort. Identifiable phases or , 
processes related to and necessary for change are research, development, diffusion, 
and adoption. Research should be assessed in its own terms. Does the research, I 
in fact, advance knowledge? The development phase involves two stages--invention ' 
and design. The two phases of diffusion are dissemination and demonstration. ! 
In the adoption process phase there are a trial stage, a process of installation, I 
and finally a process of converting the invention into a non-innovation. The I 
several phases of the change process are interdependent, and development of all i 
the phases should be emphasized in change strategy. Research provides the i 
foundational condition for the change process. Development requires that In- 
stitutions of higher education be federated. The diffusion phase has been ' 
inadequate, but Is soon to be improved by the establishment of an Educational f 
Research Information Center. Although, in the past, education has emphasized : 
the one-shot relationship in fostering adoption of inventions through contact 
between university and school personnel, new mechanisms are emerging. This ' 
is an exciting period in education because, for the first time, it is possible \ 
to anticipate that resources may match plans. (MF) ' 
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Credibility is an underlying theme i n the dirtusion of innovations. What is 
worthy of belief and who is worthy of belief are key influences in the acceptance 
or rejection of an idea. The various case studies examined emphasize the importance 
of two distinct levels of credibility. The first level is the nature of the 
innovation--the visible superiority of hybrid corn led to its rapid adoption. 
The second level of credibility lies in the nature of the person who is the change 
agent or early adopter. An example of this is the foreign status of the change 
agent who tri3d to get villagers to boil water and the failure of this project. 
The behavior of an audience toward an innovation may be considered to be determined 
by these factors: 1) the extent to which the change is congruent with the cultural 
values and perceived needs of the people, 2) the degree to which the source is 
acceptable to the public, and 3) the qualities inherent in the innovation. The 
educational practitioner as a diffusing agent may learn from implications of 
the cited case studies. Consideration of credibility with manipulation of 
communication techniques may increase the probability of adoption of an 
innovation, (MF) 
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It is not the responsibility of the college academic teacher to interpret 
his research to the practitioner--this interpretation must be cdrried out by 
others. The responsibilities of the college teacher are to transmit knowledge, 
information, and techniques in his subject area to his students and to contribute 
to the body of knowledge in his subject area. The college teacher must use the 
most advanced procedures and the most recent knowledge. If this makes him 
unintelligible to his audience, how can his findings be interpreted and applied? 
One method is for teachers to attend summer session and/or extension courses. 
Another method is to have resumes of research reports made available at convention 
sites. A good way to interpret research is to increase the number of clinics 
and workshops geared to this purpose at district and national conventions. A 
fourth method would involve state associations, state education departments, 
and state colleges and universities in clinics for elementary and secondary 
teachers. A corps of "near researchers" should be developed to counter a possible 
growing estrangement between the researchers and teachers of physical education. 
Only if these steps are taken can research be immediately applied to improve 
the teaching of physical education* (MF) 
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Changes in Army Training Programs and procedures are described to illustrate ^ 
institutional change resulting from implementation of applied research, serving 
since 1951 as a Research and Development Agency under Contract to the Army, The 
Human Resources Research Office of George Washington University has identified I 
institutional change through research utilization as resulting from implementation ' 
of specific research tasks or through consultant activities called "Technical I 
Advisory Services." Seven steps for effective utilization of research include (1) I 
Statement of the problem to facilitate research, (2) Constant Interaction between | 
the con<;umer and the researcher to clarify the problem, determine research procedures,, 
and maintain current status of information affecting direction of research, (3) , 
Communication of research results in terms appropriate for implementation, and 
(4) Continued participation by research personnel to insure most effective use of < 
the research factors influencing successful implementation of specific research | 
results include timeliness, concreteness , extent of personal interest in the value 
of the findings, quality of communications, cost, engineering, capability to 
translate findings into operational terms, and compatability of findings with long * 
accepted practices. This paper was presented at the Convention of Southeastern ' 
Psychological Association (New Orleans, March 31, 1966) and is available as ' 
AD 634 839 for $1.00 HC, $0*50 MF, from Clearinghouse for Federal Scientific ' 
and Technical Information, Cameron Station, Springfield, Virginia 22314 j 



FWL 012 nnpo 

"Facilitating the Change Process in Agricultural Education," in The Literature of 

Educational RDD&E, Vol. II, Part Two, The Oregon Studies in Educational RDD&E 
Hobbs, Daryl 
Monmouth, Oregon 
Teaching Research 

Prnj>rt Nn (1-0701, Grant Nn nFG-0>70-4977 > 197^ 

*Innovation/*Agr iculture/*Education/*School Systems/"*'CMange Agents/*Re5earch/ 
*Ulffusion/*Communication (Thought Transfer )/*Educational Innovation/*Experiment 
Stations/ 



The factors that favor innovation are more operative in agriculture than in 
educational systems. In education, the adopter is not the only one affected by 
an innovation; others in the school are affected and so are those served by the 
school. In education, there is a relative lack of formalized research and 
diffusion function for innovations. Moreover, in education there are not the 
rewards for innovation that are provided by agriculture. Nor is peer group 
influence as important among educators as among fanners. In addition, the 
material innovations that are common in agriculture are more readily adapted 
than the non-material innovations involved in education. Change needs to be 
legitimated in education by a system comparable to Agricultural Experiment 
Stations. It also needs to be legitimated with administrators, teachers, and 
the public. The change agent in education needs to concentrate on administrative 
officials and on potential adopters of the innovation. He needs to develop a 
communication strategy planned to provide the type of information needed at each 
state of the adoption process. Finally, it is possible that the farmer is willing 
to accept innovation because he has considered and accepted the idea of innovation. 
The question is raised whether school systems have come to this position. (MF) 
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Six patterns of use of scientific resources were seen in the area of social i 

practice at the Cente'' for Research on the Utilization of Scientific Knowledge. 1 

These patterns were: 1) derivation of action design from relevant research ^ 

findings, 2) the adoption of experimentally tested models of practice, 3) diffusion j 

between practitioners, 4) diagnostic team with feedback, 5) internal action- ' 

research process, and 6) the training of consumers to be open to the use of science. I 

Also, differences were discovered between the application of social research and I 

the use of the biological or physical sciences. These were: I 

1. Most significant adoptions of new educational or social practice require i 
significant chanjes in the values, attitudes, and skills of the social practitioner. | 

2. Most s-^'gnificant changes in mental hcralth or education are adaptations 
rather than adoptions of the innovations of others. ' 

3. The concept of "social invention" has not been developed adequately. j 

4. In social practice the practitioner gets very little feedback about the ■ 
effectiveness of his adoption effort. 

5. The ways in which mental health and educational practice are organized 
provide little stimulus for the practitioner to take risks in searching for ^ 
and using new resources. | 

6. The social practice fields have not developed the networks, procedures, ^ 
and manpower resources necessary to link basic and applied research to operating ^ 
practica. i 

From the studies made, it was concluded that the research utilization function ^ 
of the staff required that they be linking agents. (MP) 
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A study was made of innovativeness in teachers of vocational agriculture in 
Ohio, The study involved 101 teachers who had taught at least three years. The 
categories into which the teachers were classified were innovator, early adopter, 
early majority, late majority, and laggard. The more innovative the teacher was 
the more education he was likely to have obtained, the more he was apt to use 
impersonal information sources, the fewer his years of teaching experiences, the 
younger he was likely to be, the more likely he was to have classes for out-of- 
school youth and adults, and the greater the degree of opinion leadership which 
he was likely to hold. It was found that tho different sources providing 
information influencing the adoption of innovations varied according to the adopter 
categories but did not reach the .05 level of significance* It was also found 
that experienced teachers of vocational agriculture were influenced by different 
sources at the different stages in the innovation-adoption process. The implication 
of this finding is that state staff personnel cannot depend on one or two sources 
to lead to the adoption of innovation. Other implications of the study are listed. 
(MF) 
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Descriptions of research activities, development activities, and dissemination 
activities are preliminary to an understanding of models of relationships among 
these functions. These models, in turn, provide guidance in framing the context 
in which educational research and development are done* Research may be basic 
or applied, conclusion-oriented inquiry or decision-oriented inquiry, molecular 
or molar. Research activities in education may be described in terms of various 
academic disciplines adjacent to education, or research possibilities may be 
approached by developing some idea of the problems of education. Developmental 
activities produce materials, techniques, processes, hardware, and organizational 
formats for instruction. Like research, there are several structures which illus- 
trate development possibilities. These include age-grade level, academic disciplines 
as subject matter, and categories like instructional systems and teacher role. 
The key function of dissemination is to make information about research and develop- 
ment available in usable and effective forms. An example of a data base as a 
dissemination device is that of the Educational Resources Information Center (ERIC). 
Dissemination may occur in a variety of forms including conversation, print, film, 
tape, or slides. The models of the relationships among functions fall into three 
principal types: a linear model, a decision-oriented model, and a linkage model. 
It may be emphasized that in the main educational research and development is 
mission-oriented. (MF) 
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An analysis of the development cycle used in the Teacher Education Program at 
the Far West Laboratory for Educational Research and Development showed that about ' 
half the effort expended was on tasks that were primarily development and the other 
half on research and evaluation- That is one answer to the basic strategic question I 
of what the relative balance should be. Where there is a strong development emphasis ' 
and a weak research emphasis, improvement may be dubious and uncertain. Where the I 
development effort is slighted, the result is a product that will fail. The basic I 
question of proportion of effort can be divided into three parts: I 

1. Should a comprehensive research foundation be laid before the development , 
program starts, or can the research foundation be developed as the program progresses?, 

2. At what point must an instructional model be developed to maximum 
effectiveness? 

3. At vohat point should the development of an educational product stop? At | 
Far West Laboratories the second alternative in the first question was chosen. I 
The strategy that answered question 2 was to use just enough evaluation on the first 
minicourse developed to bring the model the desired level of effectiveness. In ^ 
answer to the third question, the conclusion reached was that a product should be ^ 
developed to the point where it is fully ready for operational use in the schools. | 
An appendix gives percentages of effort. (MF) ^ 
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Regional laboratories have be"en charged with serving as the link between so-called 
educational "research" and educational "practice," a link loosely known as "develop- 
ment." Research generates development and development generates research, with 
both leading to continued reduction of uncertainty, i.e., effort is allocated to 
different ^'uncertainty reducing possibilities" as development proceeds and knowledge 
accumulates. Activities in any product-oriented organization should be conducted 
according to certain basic assumptions: (1) The solution to a problem should be 
sought within the context of the problem. (2) The solutions to educational 
problems will be necessarily complex and many-faceted. (3) A successful solution 
requires the presence of certain essential conditions including people who accept 
the solution and are motivated by its evidence. (4) Precise assessment is funda- 
mental to any R and D program. (5) A profession of educational engineers or 
developers must be trained. (6) The educational gatekeepers must not be subverted 
in the performance of their elected or assigned duties. An R and D staff, because 
it is a group of specialists working together within an independent institution, 
contributes to its own training and that of others through a climate which provides 
security, individual identity, and coordination of efforts. 
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The research function has an Intergral role in the process of educational \ 
development. In describing this role, several assumptions are made. First, it 
is assumed that the current interest in educational research and development is 
aimed at enhancing the effectiveness of education. A second assumption is that ! 
both the educational needs and the educational resources of a democratic society ' 
are complex and changeable and entail multiple and conflicting human values. A \ 
third assumption is that educational development consists of many forms of invention i 
of solutions to educational prob"!ems. Finally, the assumption is made that 1 
educational research entails a number of forms of inquiry aimed at knowledge about , 
education and at the orderly and economical representation of that knowledge* , 
Development in education must proceed with a paucity of available scientific know- 
ledge and technology. To do this, it is necessary that research roles be built I 
into development. By research roles in development is meant any and all inquiry | 
needed to provide knowledge, to design solutions, to determine applicability, I 
and to bring solutions into the field. Research in a development project must 
provide the information necessary to make effective decisions. Quite typically, ' 
the quantity of information needed is enormous. When the function of development ' 
is assumed by a university, a close relationship of research to development is | 
particularly desirable- (MF) 
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To build a sound bridge from the experimental studies of learning to the 
classroom, a series of steps are needed. The three steps in basic-science research 
are: research on learning with no regard for its educational relevance, research 
on learning which is not concerned with educational practices but which is more 
relevant than that of Step 1, and research on learning that is relevant because the 
subjects are school-age children and the material learned is school subject matter 
or skill. Applied-science research can be divided into three steps, too: research 
conducted in special laboratory classrooms, with selected teachers and few 
students; tryouts of the results of prior research in '^normal" classrooms with 
typical teachers; and, finally, developmental steps related to advocacy and 
adoption. A strategy of innovation is also needed. Until recently, little of 
this kind of effort went into the problems of providing better teaching 
materials and tested instructional methods. Now we have a number of experiences 
of 1 arge-scale cooperative efforts in producing better teaching materials through 
the collaboration of subject-matter experts, learning specialists, and teachers. 
In addition, large regional research centers have been established by the U. S. 
Office of Education. It will be of interest to study their output. (MF) 
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A model is presented of research and development as seen and understood from ^ 
the point of view of a sponsor or user. Developed as an alternative to the Guba- 
Clark model, this model is based on the conviction that research, development, 
and school operations are different kinds of activities with quite different 
objectives or outputs. The model is expressed in three planes: one plane 
symbolizes the knowledge orientation of research, a second plane symbolizes the 
process orientation of development, and the third plane symbolizes the activities 
characteristic of school operations. A "walk-through" of the model is illustrated by 
a life situation in which a school official, realizing that certain outputs desired 
by society are not being achieved, sets in motion initiatives and outputs in each 
plane of the model. Other interactions are possible with other life situations. 
Awareness of interaction between the activities represented by the model places 
constraints upon the professional behavior in each activity and emphasizes the 
problem of information flow. The most significant aspect of the model may be its 
attempt to represent each of the activities in terms of particular kinds of outputs. 
If schools are graded by their output, they may begin to call for the kind of 
research and development that will provide the basis for improvement. (MF) 
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In current practice the approaches to research and design are almost identical , 
yet the purposes of design and research are different; and analysis, which is 
essential to research, restricts the effectiveness of the design solution. Therefore, 
a methodology other than that used for research may be more appropriate for designing 
and engineering projects. An empirical ten-step design strategy has been suggested 
by several engineers and designers. The steps are these: function determination, 
ideal system development, information gathering, alternative systems suggestions, 
selection of the feasible solution, formulation of the system or solution, review 
of the system design, test of the design system, installation of the system or 
solution, an establishment of performance measures. Empirical evidence shows that 
this design approach has produced better results for design projects than the 
conventional research approach. Four different avenues of activities may be followed 
'n investigating the problem of the design of a design system. One relates under- 
graduate teaching to ongoing design projects of laboratories and professors; another 
involves the decision-making process; a third concerns research into creativity; 
and a fourth involves developing innovative alternatives to design. In a pilot 
>tudy, outstanding practitioners of several professions were interviewed. Strong 
similarities were found in the actual strategies of those interviewed. It was 
concluded that the design approaches of various professions could be translated 
into one general design model. (MF) 
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An empirical rationale for policy management of educational research and ~i 

development is presented here. Management of educational research and development . 
is affected by what a sponsor considers them to be and by what he considers their 

goal to be. The reason for managing research and development for education is the I 
reason for managing anything to achieve the objectives set for a program at acceptable' 
levels of financial and manpower cost and within the desired time limit. The followinig 

kinds of activities are embraced under the general heading of managing research and I 

development for education: I 

1. Identifying the overall goal and clarifying basic assumptions i 

2. Identifying the priorities i 

3. Identifying research and development goals 

4. Identifying specific objectives ' 

5. Choosing among alternative project and program activities in terms of I 
service to goals and objectives ; 

6. Implementing and monitoring specific projects and programs 

7. Developing and sustaining communication networks to insure appropriate * 
and adequate information flows for planning purposes ' 

8. Developing appropriate data input mechanisms for planning and feedback ' 
mechanisms for program evaluation ^ 

9. Providing, identifying, and recruiting supplies of appropriately trained 
manpower , 

10. Evaluating the impact of research and development in terms of the overall 

goal of the program. ' 

This description of management of educational research and development is based ^ 

on recent U. S. Office of Education experience and upon interviews with important « 

policy makers in educational research and development. (MF) i 
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Co-sponsored by the National Academy of Sciences and the bocial bcience Kesearch 
Council, a survey was made of the behavioral and social sciences. Half of the amount 
spent by the Federal government in 1967 on basic and applied research in these fields 
went to universities. The universities divided up their money into three roughly 
equal parcels, one of which was spent in organized research administered through 
the disciplinary departments, another through the professional schools, and the final 
third through research institutes, centers, and laboratories. The impression was 
gained through the survey that the typical Ph.D-granting departments and the profes- 
sonal schools especially prided themselves on their basic research; whereas, the 
institutes were interested in pressing social problems. The suggestion is made that 
there should be a distribution of prestige along the spectrum from basic to applied 
research and development. Also, a proposal is made for a new type of organization 
yithin the universities, so that they might better fulfill the mission of public 
service to which they might aspire. Behavioral and social scientists, as a body 
should be doing work that is pertinent to social problems. Even now, their ac- 
complishments are by no means trivial. The problem of what is basic research, 
what is applied research, and what is development is discussed, (MF) 
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A study was made of the organizations in the National Program of Educational 
Laboratories. This program includes 15 laboratories, nine university-sponsored 
research and development centers, and several federated projects known collectively 
as the National Laboratory on Early Childhood Education. Characteristics observed 
are the ""ol lowing: 

1. There is an increasing tendency to concentrate efforts on a closely 
defined set of objectives or problems. 

2. The centers tend to emphasize problem-oriented research; the laboratories 
tend to emphasize development of instructional systems or other products. 

3. The development of systems to facilitate learning is the primary or 
ultimate goal of most laboratory and center programs. 

4. The centers and laboratories exhibit lively concern for the improvement 
of teaching and teacher education. 

5. Attempts are made to give attention to every element and condition 
likely to affect significantly the results achieved. 

6. The new research and development agencies accept responsibility for a 
continuing process of modification and refinement until the desired educational 
objectives are obtained. 

7. The building of organizational links to facilitate the flow of knowledge 
into educational practice is going forward slowly but persistantly. 

Urgently needed are: a stepping up of the support level and the establishment 
of conditions conducive to autonomy and accountability. It is believed that the 
new research and development organizations promise to supply essential ingredients 
for continuous improvement of education. (MF) 
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Companies do not grow and prosper merely by having more creative people in 
them. There is very little shortage of creative people in American business. 
The problem is that these people do not make the right kind of effort to implement 
their ideas. This is a form of irresponsibility. There is even a relationship 
between creativity and irresponsibility. Few of the ardent advocates of creativity 
in business have a day-to-day responsibility for implementing ideas in a business 
organization. Very often the critics of conformity in business are outsiders to the 
central sector of the business community. Within business organizations, the most 
creative men (below the executive level) criticize conformity and are known as 
corporate malcontents. The reason the executive so often rejects their ideas is that 
he is a busy man, and ideas create problems. When an idea is offered to him, it 
should be documented. How thorough the documentation needs to be depends upon the 
situation. Organization is hostile to creativity and innovation because they disturb 
the order that is necessary for it to get the job done. A company cannot function 
as an anarchy, and although Parkinson is entertaining, rules that look foolish may 
not be. There is a possible need for making creativity yield more innovation, and 
several companies have devised methods for this. (MF) 
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A series of 17 generalizations describes the educational research enterprise in [" 
the United States under four headings: Organizational patterns, training programs, 
dissemination techniques, and the relationship of research to practice. Eight 

summary statements outline the primary implications of these generalizations and I 

define educational research in the United States as loosely organized, university ^ 

based, individually directed, theory oriented, committed to experimentalism, in a t 

psychostatistical tradition, a part-time pursuit, and Federally funded. Related 1 

problems include low utilization of research by practitioners, inadequate I 

mechanisms linking the worlds of the researcher and the practitioner, inadequate , 

programs for training educational researchers, and a lack of tools and strategies , 
for the adoption of educational improvements based upon research. Alternatives 

for a redevelopment of educational research include the development of new research ' 

structures to complement existing structures, the building of effect -e linking i 
mechanisms between the research and practitioner communities, and the development 
of new training programs. 
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Educational research is expanding in size, resources, and in the concept itself. | 

What is research and how may it affect educational practice? First, it is clear | 
that the classic models of experimentation cannot handle the full range of 

educational inquiry. Second, there is a developing interest in establishing better I 

linkages between research and practice. Finally, there is a shift away from ' 

questions of techniques and methodology in research to other aspects of the research ! 

activity. Objections to experimental ism are the intervention of the investigator I 

and the difficulty of defining the relevant variables a priori in a classroom. I 

An alternative is a experimentalism, in which real life is studied. A promising ^ 

tool for this kind of inquiry is the computer. Between research and practice there | 
are a whole series of emergent functions. One of these is development, which is 

concerned with identifying problems, inventing solutions to problems, engineering I 
the proposed solutions into practical form, and field testing the packages. Oiffusior] 

is another function. It informs about packages and programs and demonstrates them, : 
offering training in their use, and assisting in their installation and servicing. 

Another dimension along which educational research is expanding is the development ' 

in the research process of non-methodological aspects--the problem, the objectives, ^ 
and the theoretical framework. It seems likely that the new challenges in research 
will be met by the research community. (MF) 
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Traditionally it has been assumed that there is a fairly smooth sequence from 
research through a developmental phase to the utilization of research results. 
Evidence shows that this sequence is seldom followed in actual practice and that 
special efforts must be made to assure that the results of research are applied. 
In an attempt to remedy this problem in education, a traveling seminar and a 
conference were organized for the implementation of educational innovation. 
Schools across the U.S. where significant innovations had been introduced and 
in operation for at least one year were selected. Groups composed of local 
administrators. State education department officials, and college teachers 
toured the schools, after which they participated in a conference. One year 
later the innovative activity in districts participating in the tour was compared 
with that in a nonparticipating control group; it was found that the participants 
had introduced more innovations than the nonparticipants. Analysis of this 
experiment suggests that such programs can speed research into application more 
successfully than printed communication or demonstration schools. Factors leading 
to the successful application of research results and the importance of the regional 
laboratories in evaluating the effectiveness of innovations are also discussed. 
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Educational research and development operations in the past decade have in- 
troduced new approaches to change and have demonstrated the potential for con- 
tinuing improvement. These operations a^e characterized by the recycling of 
processes until intended effects are achieved, by the attention paid to all tye 
major elements in learning environments, by the linking of many organizations and 
institutions in the implementation of programs, and by the accentuation of 
positive elements in the social and cultural environments of disadvantaged popu- 
lations. But research and development organizations alone cannot bring about 
needed improvements in education. The adopting agencies--schools and colleges-- 
must play their part by using appropriate problem-solving processes to specify 
and achieve the desired outcomes, "On the scene" observers agree that application 
of systematic, large-scale research and development processes to education started 
at near zero capacity, that significant progress has been made, and that additional 
measures are required. One measure should be the establishment of a National 
Institute of Education, Educational research and development organizations have 
already contributed a variety of products to education, have refined their strategies, 
and have made a modest beginning toward building linkages. If the indicated J 
corrective measure? are taken, the next ten years should see a raising of the level | 
of educational achievement, (MF) , 
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This study was initiated in 1964 under the sponsorship of the Cooperative 
Research Program of the Office of Education to develop and supply data useful 
to policy makers and planners of training programs for research, development, 
and diffusion personnel in education. The first task was the construction of 
a logical structure to account for the formally defined institutional roles, from 
which a picture of the research community in 1964 could be drawn and future demand 
projected. Chapter 2 provides a general description of the community in 1964, 
analyzing it by institutional settings and researchers* roles, estimating the 
actual number of researchers and the number being added to the community. Chapter 
3 deals in detail with the procedures used, funding projections, and personnel 
projections. Chapter 4 considers the effects of curtailment of funds for ESEA 
Title IV research training programs which will reinforce the existing pattern 
rather than enable new centers for training and roles for researchers to be 
established. Finally an attempt is made to define the roles which could be 
expected to emerge under the new research, development, and diffusion funding 
programs, namely director and staff of outside-funded development programs, 
technical support personnel, development project personnel, training personnel , 
and stimulators and coordinators of activities. Eight appendixes give additional 
documentation. (MBM) 
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The Office of Education is interested In the continuing, improvement of the 
American educational system. To implement this interest, it is involved in 
1) the generation of knowledge about learning in education, 2) the development of 
validated and economically feasible instructional products, and 3) the dissemination 
of information that will enable local schools to be aware of and put into use the 
new knowledge and techniques. As a necessary concomitant, the Office of Education 
is also involved in the training of research personnel to assist with these 
activities. To achieve maximum gains from the expenditures of available funds 
and manpower, it is obligated to manage research. Procedures are established, 
priorities are determined, equitable distribution is attempted, and coordination 
with other Federal agencies is provided. Dissemination is encouraged by the 
Office of Education through the Educational Resources Information Center (ERIC) 
system and through regional laboratories. Cooperative Research investment in 
ERIC activities amounted to approximately 2.7 million in 1967 and the regional 
laboratories estimate dissemination at about a fourth of their total budgets. 
Definitions are given of research, development, demonstration, dissemination, 
and training of researchers. A description is given of the relationship of these 
components to each other in the process of effective change. (MP) 
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Evaluation studies are to be judged by their usefulness to decision-makers. 
Differing in purpose from research studies, they need not be subject to the same 
criteria. Concerns with value and utility are generic to evaluation but do not 
play as Important a role in research. The symbiotic relationship of the two 
forms of study is illustrated by a summary of an evaluation study of the Head 
Start program at New Nursery School in Colorado. The first difficulty of the 
evaluation lay in matching specific behaviors of the children with objectives 
of the program. Next, there was the difficulty of finding or devising appropriate 
measurement procedures. Because of these difficulties and others, an evaluation of 
this evaluation study, applyi ig the conventional standard of research work, would 
probably find that the study /as poorly done and that drawing conclusions from it 
was next to impossible. Yet within the framework of statistical decision theory 
and as illustrated in an assumed situation, the New Nursery School evaluation 
may have "pay-off" value. In the illustration it helps a superintendent who 
wishes to recommend Head Start to his school board. When a desirable but uncertain 
outcome needs to be estimated the evaluation study may be valuable, (MF) 
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This presentation clarifies the role ot evaluation in pudnc schools and discusses 
major problems which block the effectiveness of the role. Evaluation is used in 

schools to produce information that helps decision makers choose among alternative ' 

courses of action. Purpose, rather than methodology, distinguishes evaluation ^ 

from research. The need for care, precision, and logical thought, however, is t 

common to both. One set of major problems is technical and involves such things | 

as measurement and choice of appropriate analysis techniques. A second set I 

revolves around the need for decision makers to follow a rational approach and | 

to avoid reverting to "gut" feelings in every stress situation- Finally, ^ 

evaluators must produce pertinent information, in the most useful format, at i 
key points in time. 

I 
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The application of a PERT system to a large-scale project will increase the 
probability of accomplishing project objectives by providing greater visibility 
of (1) the project objectives, (2) the relationships among the parts of the 
projects, and (3) the relationships of those component parts to the project 
objectives. Consequently, educational researchers should find that a PERT 
system will increase their collective ability to coordinate planning and evalua- 
tion efforts • A review of a PERT system applied to a study of disadvantaged 
school children indicates that in order to achieve optimal utility, the system 
should be implemented well ahead of the project starting time, and must be 
responsive to program developments through continued updating procedures. 
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Originally developed as a management technique, the Program Evaluation and 
Review Technique (PERT) may be applied to educational projects. Since the initial 
purpose of PERT was to probe areas of uncertainty and to discuss possible alterna- 
tive paths, the use of a model network in education is somewhat hazardous. 
Nonetheless, several model networks are given for the selected areas and methodologies 
of 1) experimental research, 2) survey research, 3) historical researcl., and 4) 
curriculum projects. A PERT network is further illustrated by breaking down the 
various parts of a doctoral dissertation. A major area not applicable to the PERT 
network is that of maintaining classroom activities. PERT deals only with the time 
constraint and does not include the quality, quantity, and cost involved in a 
project. As a research tool it is designed to evaluate progress, focus attention 
on problems, determine completion dates, provide frequent status reports, and 
predict the likelihood of reaching stated objectives. Its advantages include the 
creation of a realistic, easy-to-communicate plan; simulation of alternative 
plans and schedules; and the utilization of all possible resources. PERT came 
about as an answer to the need of the Polaris Project and was based on the 
addition of probability tc the Critical Path Method (CPM). (MF) 
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So that the future of research and development impact on education is brighter i 

than the past, the Far West Laboratory for Educational Research and Development, | 

like similar non-profit organizations, must consider how best to disseminate , 

information about its products and how to facilitate product distribution and i 
utilization. Part of the Laboratory's consideration is that of policy, a 

determination of the scope of its responsibility for these activities in relation I 

to its objective of creating opportunities for children to learn. Another ' 

consideration is that of constraints. Yet another concern is that of the kinds I 

of persons or models that might link products and product users. Because of the i 

Laboratory's product orientation, the commercial marketing model has been i 
considered for use. Consideration of marketing strategy includes analyses of 

the market target, the market structure, and the product. Choice of a marketing ' 

plan involves the question of what distribution channels to use. Another question t 

arises with respect to the choice of agents for distribution. The next question : 

to arise is that of what the market policy should be. Factors that need to be , 
considered are priorities, resource allocation, choice of producer and consumer, 

control and standards, market research, and product policy. The marketing model ^ 

is especially useful in that it calls for explicit consideration of the consumer. | 

It also raises the basic question as to the extent that marketing considerations j 

should influence Laboratory design and development policies. (MF) i 



"Incorporating the Products of Educational Development Into Prac^ice;^ in Literatu 
of Educational RDD&E, Vol, II, Part Two, The Oregon Studies in Educational RDD&E 
Becker, James W. 
Monmouth, Oregon 
Teaching Research 

Project No. 0-0701 , Contract No, 0EG>0-70-4977 , 1972 



^Educational Development/*Laboratories/*Research and Development Centers/*Individual1z4d 
Tnstruction/*Demonstration Programs/*Mathematics Materials/*Elementary School 
Mathematics/*Field Studies/*Diffusion/*Feedback/*Project Training Methods/*Program 
Evaluation/*Evaluation Criteria/ 



An example of how an educational laboratory moves a product into wide-scale 
practice is the experience of Research for Better Schools, Inc. (RBS) with 
Individually Prescribed Instruction (IPI). RBS first determined that the need 
most felt by educators was for individualized instruction. Then RBS staff 
developed criteria with which to identify a program that, meeting this need, 
would be of interest to develop, field test, and diffuse. IPI met the RBS 
criteria and was adopted. Two other organizations have cooperated with RBS to 
move the program from theoretical to productive development. RBS responsibilities 
are to field test, evaluate, and build the management and training procedures. 
In Phase I of RBS effort, six schools have been selected as demonstration- 
developmental schools. Agreements have been reached with the schools, staff have 
been trained, monitors (field engineers) now work with the schools, and data 
is constantly being collected and appraised. Phase II is a wide-scale, carefully 
controlled expansion of the IPI Math System in a variety of environmental settings. 
An unusual number of problem areas have been met by RBS in this phase. Several 
methods for achieving wide-scale adoption are being considered: request for 
proposals, straight coirmiercial ism, and a laboratory approach. Examples are 
given of adoption procedures used by other educational laboratories. (MF) 
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^'Limitations of Systems Analysis," in The Literature of Educational RDD&E, Vol. lU 

Part Two, The Oregon Studies in Educational RDD&E 
Hartley, Harry J* 
Monmouth, Oregon 
Teaching Research 

Project No. 0-Q7QK Contract No. OEG-0-70-497 7, 1972 

^Systems Analysis/*Analytical Criticism/*Educational Planning/*Systems Approach/ 
*Education/*Problems/*Definitions/*Educational Administration/*Environmental 
Influences/ 



Although systems analysis has shortcomings in its application to education, [ 
the limitations are outweighed by the potential advantages to be gained. The . 
four areas of application of systems analysis in education are in policy formulation, 
management, instruction, and research* Most of the literature describing systems I 
analysis is long on persuasion and short on critical appraisal. The result is to ' 
make it difficult for educators to judge the relative worth of competing systems ! 
techniques, of which there are 60 or more. To remedy this, 25 limitations of I 
systems analysis are listed: confusion over terminology, problems in adapting ( 
models, a wisdom lag, illusions of adequacy by model -builders, inadequate impetus , 
from states, centralizing bias, unanticipated increased costs, goal distortion, j 
measuring the unmeasurable, cult of testing, cult of efficiency, spread of institu- 
tional racism, political barriers, conventional collective negotiations procedures,' 
lack of orderliness for data processing, monumental computer errors, shortage of | 
trained personnel, invasion of individual privacy, o>^ganizaticnal strains, resistance 
to planned change, antiquated legislation, biased evaluation, image problems, defects 
in analysis, and accelerating rate of social change. It is proLable that many of ' 
the present limitations can be overcome as more persons apply their talents to this > 
topic. In the final analysis, the success of systems procedures is dependent upon | 
the artistry of the user. (MF) ! 



FWL 012 0087 



''The Systems Approach to Education: Mystique and Reality," in The Literature 

of Educational RDD&E, Vol, II, Part Two, The Oregon Studies in Educational RDD&E 
Carper, Launor F. 
Monmouth, Oregon 
Teaching Research 

Prnifict Mo. 0^0701. Contract No. 0EG-0>70-4977 , 197 2 



*Systenis Analysis/^Systems Approach/*Training Techniques/*Project Training Methods/ 
*Education/*Military Air Facil ities/*Systems Development/*Mexican Americans/ 
*Minority Group Children/*Evaluation/*Reading Skills/ 



Systems analysis was used in the development of two different training programs; 
One program involved the development of a training system for one of the country^s 
major air defense systems. The other program dealt with the problem of developing 
a better approach to the teaching of introductory reading to Mexican-American 
children. The development of the military program was based on a long history 
of experience in developing similar programs. The program placed major emphasis 
on the careful simulation of all of the inputs to the air defense system. Emphasis 
was also placed on the idea that the crew had to work as an integrated unit. 
The third major concept concerned the careful recording and analysis of individual 
and crew performance, and the feedback of the analysis to the crew. In the reading 
program for Mexican-American children the characteristics of the model used were: 
first, an empirical approach involving evaluations and revisions of procedures; 
second, the conduct of research and development activities in an actual school 
setting; and third, the active participation of regular classroom teachers and 
school administrators. To an extent that varied, each of eight system analytic 
steps were followed in the two developmental programs. The steps were: (1) 
stating of the read need, (2) defining educational objectives, (3) defining 
constraints, (4) identifying and analyzing alternatives, (5) selecting the best 
alternative, (6) implementing for testing, (7) evaluating the system, and (8) 
providing for feedback and modification. In summary, systems analysis is a point 
of view and a set of procedures; it is not a guarantee of success. It has a 
particular applicability in education. (MP) 
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Development, Diffusion, and Evaluation , Also in The Literature of Educational RDD&E, 

Vol. II, Part Two, The Oregon Studies in Educational RDD&E 
Cuba, Egon G. 
Bloomington, Indiana 

National Institute for the otud> of Educational Change 

October, 1967, 43 p, 

*Deve1opment 5 Information Svstems/*Diffusion, Evaluation, Change Agents/*Educational 
Practice/*Research Util ization/*Educational Research, Researchers, Research and 
Development Centers, Regional Laboratories, Educational Innovation, Evaluation 
Techniques, Portland, Title III Projects/ 

The knowledge gap between initial research and final use is discussed in terms of 
four states of the theory-practice v^jntinuum (Research, Development, Diffusion, and 
Adoption). The two middle stages are emphasized. Research and Development centers. 
Regional Educational Laboratories, and Title III Projects are suggested as agencies 
responsible for the development function* Failure of the diffusion function to 
operate satisfactorily is attributed to the lack of an acceptable strategy. Such a 
strategy would contain these elementS""(l ) Assumptions concerning the nature of the 
practitioner who will be exposed to the strategy, (2) Assumptions concerning the 
end state in which rne wishes to leave the practitioner, (3) Assumptions about 
the nature of the agtncy or mechanism carrying out the diffusion activity, and (4) 
Assumptions concerning the substance of the invention. The concept of evaluation 
is changing rapidly, and the methodologies currently jsed for evaluation need 
replacement. Traditional evaluation has four characteristics which account for 
its limited ut.lity--{l) Evaluation data are usually available only upon the 
termination of the evaluative period, (2) Evaluative data typically afford only 
a retrospective view, (3) The assumptions on which evaluative designs are based 
impose a series of constraints on the evaluation, and (4) The conistraints imposed 
create a laboratory condition within which the treatment is tested. This paper 
was prepared for the UCEA Career Development Seminar (17th), cosponsored by the 
University Council for Educational Administration and the University of Oregon 
(Portland, Oregon, October 22-25, 1967). 
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The Oregon Studies in Educational Research, Development, Diffusion, and Evaluation. 

Vol. Ill: Conceptual Frameworks for Viewing Educational RDD&E. Final Report 
Schalock, H. Del & Sell, G. Roger, (Eds.) 
Monmouth, Oregon 
Teaching Research 

Rrant. Nn. nFG^n-7n>4q77 . 1Q7? 

♦Conceptual Schemes, Diffusion, Educational Change, Educational Demand/*Educational 
Development, Educational Improvement, Educational Innovation, Educational Needs/ 
♦Educational Research/*Evaluation, Evaluation Methods, Literature Reviews, Problem 
Solving, Research Methodology, Research Utilization/ 



This volume represents the output of a year-long effort to clarify and firm the 
conceptual base that underlies educational RDD&E. It contains three commissioned 
papers authored by Drs. Hendrik D. Gideonse, Gene V. Glass and Blaine R. Worthen, 
and by Leslie J. Briggs and the one paper prepared by H. Del Schalock and 
G. Roger Sell of the staff of the Oregon Studies. The volume also contains a formal 
critique of each paper, an author res[.onse to each critique, and papers as a set. 
Individually, the papers deal with one or more aspects of the domain of educational 
RDD&E. Collectively, they deal with tne domain as a whole. As a set, the conceptual 
papers and their critiques are intended to serve as (1) a benchmark document with 
respect to the conceptual structures underlying educational RDD&E, (2) a primary 
reference for persons entering the field of educational RDD&E, and (3) a stimulus 
to the continued dialogue that these papers have only begun. Related documents 
are EA 004 582-484 and EA 004 586-589. 
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Development of a National Register of Educational Researchers 
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Bargar, Robert and others 

Columbus, Ohio 

Ohio State University 

December. 1965. 149 p.. fnRP-E-014) 

♦Educational Research, Reference Books/*Researchers/*Indexes (Locators), Question- 
naires, Personnel, Research Skills, Columbus, Ohio/ 



The primary purpose of the study was the development of a national register of 
educational researchers. The objectives were (1) To identify the population of 
educational researchers in the United States, (2) To obtain in^'ormation from 
these individuals on their personal life, educational history, professional 
identification, professional employment, research areas, and (3) To process 
the information so as to permit a multiplicity of use. The major procedural 
steps will (1) Develop a list of 17,000 names and addresses of educational 
personnel using published sources and direct contacts with research organizations, 
(2) Develop a questionnaire and necessary accompanying materials, (3) Circulate 
12,000 copies of the questionnaire, (4) Select those individuals to be included 
in the register, and (5) Process and analyze data from the questionnaire. The 
"Phi Delta Kappan" will include the first results of the study. 



Ohio State University 
Research Foundation 
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Investigation of Factors Influencing the Training of Educational Researchers ^ 

Bargar, Robert R., Okorodudu, Corahann P., Dworkin, Edward P. ' 
1314 Kinnear Road. Columbus, Ohio 43212 , i 
The Ohio State University Research Foundation ' 

MaY> 1970, £0-6-10-146> 119p. I 

♦Educational Researchers/*Professional Training/Experimental Programs, Educational ^ 
Research, Developmental Programs, Manpower Development, Graduate Study, Undergraduate! 
Study/ ! 

I 

L 

The purpose of this study was to investigate educational and occupational f 

factors which influence educational researchers and to determine the relationships | 

of the factors to the training and development of educational researchers. ^ 

Available literature on training is reviewed and summarized and an index of topics . 

related to research and career development is presented. Experimental undergraduate ' 

and graduate training programs were developed in an attempt to study the problem ! 

of recruitment and training. Designs of the experimental programs are described I 

with background, objectives, and expected output information. Evaluation data I 

derived from the first experimental undergraduate group of fall 1967-1970 includes: i 

(a) procedures and results of the first year of the program, (b) data about i 
employment of students in research, development or evaluation, (c) a survey of 

school personnel to determine potential growth and development of the program. ' 

Continuous evaluation, curriculum development, instrument development, and | 

research goes on relative to the experimental programs of the project. Recom- • 
mendations for graduate programs include development of placement criteria 
and methods of measurement and adoption of a systems approach to instruction. 

Results of the undergraduate experiment point to some necessary ingredients ' 

for empirical inquiry. References and an appendix of questionnaires and | 
data analysis results are included. (6S) 
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The Scientists Role in Society: A Comparative Study ' 

Ben-David, Josep!, ' 
Englewood Cliffs, New Jersey ' 
Prentice-Hall, Inc. • 

1971 (Paperback in the Foundations of Modern Sociology Series) | 

*Diffusion/*Financial Support/*Historical Reviews /^Natural Sciences/*Organization/ ^ 
*Policy/*Research/*Role Theory /^Scientific Enterprise/*Science History /*Science ' 
Laboratories /*Scientific Research/*Scientists/*Social Change/*Social Values/*SociologV/ 
*Uni vers ities/*Wes tern Civilization/ i 

I 

This book describes the emergence and development of the social role of the scienjtist 
and the organization of scientific work. Topics dealt with are: (1) the social rolej 
of the people who created and transmitted scientific knowledge in the traditional 
societies of antiquity and the middle ages; (2) the problem of why the study of natural 
science had never, during these periods, become a distinct intellectual role; (3) ' 
the conditions which lead to the separation of the role of the scientist from other I 
intellectual roles in the 17th century Europe and and to the professional ization of I 
science in the 19th and 2Dth centuries; (4) the principal stages in the development | 
of the organizations of scientific work, such as the scientific academies of the 17th, 
and 18th centuries, the scientific university of the 19th and 20th centuries, and the, 
research institutes and industrial laboratories starting in the last decades of the 
19th century. 

The subject is treated from a historical and comparative perspective, with emphasps 
of explanation of changes and turning points, rather than on the interpretation of . 
the ways in which scientific work is conducted. The framework is institutional and 
marco-sociological. There is an attempt to conceptualize the effects of the emergence 
and diffusion of scientific activity in modern societies. . • 

For those concerned with educational R&D training, the book is recommended for its 
historical perspective on the scientists role and the organization of science. Chapter 
8, the professional ization of research in the United States, and Chapter 9, Conclusions 
are especially relevant to understanding science in the U. S. today. Chapter 9, ^ 
should be read by those concerned with educational R&D policy. ^ 

The book is an excellent source of references. (PDH) 
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Training for Educational Research 

Buswell» Guy T, and Others 
Berkeley, California 

University of California, Center for Higher Education 

161 p. 

♦Educational Research, Educational Programs/*Postgraduate Education/*Improvement 
Prograins/*Consultants/*Doctoral Degrees/*Conferences , Questionnaires, Program 
Evaluation, Interviews, Graduate Surveys, Berkeley, California, Center for the 
Study of Higher Education 



The factors which lead to productivity in educational research were studied. The 
program was divided into four sections--(l) An analysis of questionnaires from 816 
persons receiving doctoral degrees in 1954. (2) An analysis of a previous study of 
1,750 persons receiving doctoral degrees in 1964. (3) A study of dissertations of 
1,598 persons receiving doctoral degrees in 1964. (4) A study of the background, 
training, and various other aspects of 31 outstanding scholars, Inteviews were 
also conducted to study the general climate and conditions under which productive 
scholars work. Before final recommendations were submitted, 14 consultants were 
invited to offer criticisms and suggestions at a 2-day conference. Twelve final 
recommendations were made and accepted. The conclusions indicated that research 
should be problem centered and conducted by interdisciplinary teams and can 
attack major problems with the intellectual and scholarly resources of an entire 
university brought to bear on the problem being studied. 



University of California 
Center for Higher Education 
Berkeley, California 
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A Report on Educational Research, Developmt Vf, anOlffusion Manpower, 
1964-1974 

Clark, David L., Hopkins, John E. 

Bloomington, Indiana 

Indiana University Research Foundation 

1969 ^ 

*Educational Development, Experimental Programs, Personnel Needs/*Research and 
Development Centers, Research Directors, Research Problems/*Research Projects/ 
♦Researchers, Research Utilization, Training Techniques/ 



This study was Initiated in 1964 under the sponsorship of the Cooperative Research 
Program of the Office of Education to develop and supply data useful to policy 
makers and pi runners of training programs for research, development, and diffusion 
personnel in educa^-^on. The first task was the construction of a logical structure 
to account for the formally defined institutional roles, from which a picture of 
the research community in 1964 could be drawn and future demand projected, Chapter 
2 provides a general description of the community in 1964, analyzing it by institu- 
tional settings and researchers' roles, estimating the actual number of researchers 
and the number being added to the community. Chapter 3 deals in detail with the 
procedures used, funding projections, and ESEA Title IV research training programs 
which will reinforce the existing pattern rather than enable new centers for training 
and roles for researchers to be established. Finally an attempt is made to define 
the roles which could be expected to emerge under the new research, development, 
and diffusion funding programs, namely director and staff of outside-funded 
development programs, technical support personnel, development project personnel, 
training personnel, and stimulators and coordinators of activities. Eight 
appendixes give additional documentation. 
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A Study of the Operational Feasibility of the Apprenticeship Work-btudy Program 

for Multi-Site Tryout and Evaluation 
Melnotte, Judith M., Millstein, Eugene J., Dunn, James A. 
P. 0, Box 113, Palo Alto, California 94302, 
American Institutes for Research 

March 197?, OEG-0-71-4711 , 79 p, ^ 

♦Research Apprenticeships/*Work Experience Programs/*Transfer of Tralnlng/*Eclucational 
Researchers/*Research Needs/Aerospace Industry, Career Change, Unemployed, Inservice 
Education/ 



This report evaluates a pilot project to crosstrain unemployed aerospace personnejl 
in educational research and development. {The project is described in another document. 
Development of an Apprenticeship Work-Study Program Model for the Cross-Training of ' 
Physical Scientists for Work in Educational Research and Developments) The evaluation' 
explores 14 questions, such as "How successful is the program? How general izable or I 
exportable is the model? What modifications of this model .might be considered in I 
future efforts? The evaluation was based on analysis of recruitees, data from i 
application forms and interviews, a standardized pre- and post-test of trainees, | 
weekly chronicles of activities and reactions, self-ratings, staff ratings, interviews 
with trainees and staff, follow-up interviews with trainees, and the employment recordj. 
Trainees and staff agreed that the training produced transfer of old problem-solving 
skills to the new discipline and reoriented the trainees to educational research and i 
development. Three of the six trainees found jobs in the field. The training model » 
itself was judged effective and replicable. Factors affecting its successful use are' 
enunciated. Some changes of emphasis and procedure are also suggested. The report * 
notes that demand for educational researchers has slackened since the project was 
conceived. Three ways to modify the training model are therefore suggested: choose ■ 
different trainee populations, integrate it with a university curriculum, or adapt ' 
it for inservice education. Modification of the model for inservice education is ' 
advocated. (AS) ^ 
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Columbia University 
Teachers College 
New York, New York 



The Development of Professional I ersonnel in Educational Research, Volumes I and II ' 

Millikan, Nancy H. • 

New York, New York i 

Columbia University, Teachers College " 

1967 > 7q 2 p., (Report No, BR-5>8068) ' 

♦Educational Research/^Graduate Study/*Insti tutions/*Educational Resear^.iers , ^ 

Doctoral Degrees, Literature Reviews/^Professional Training, Research Apprentice- ' 

ships. Admission Criteria, Objectives, Manpower Development/^Research and Develop- I 

ment Centers, Individual Characteristics, Financial Support, Career Chc'ce, i 

Statistical Analvsis. New York City/ i_ 

This study was undertaken to identify opportunities that may exis'* for research i 

preparation in graduate institutions. To examine the institutional and training . 
arrangements that may be related to output of researchers, and to investigate 

the potential commitment of recent doctoral recipients to educational research. I 

Data for the study came from two major sources--(l) Data collected by the writer, ' 

including content analysis of the catalogues of 110 graduate institutions, case ' 

studies of selected research organizations, and interviews with 20 individuals, i 

and (2) Data already existing, including institution surveys, questionnaire surveys | 

of behavioral scientists, and questionnaire surveys of the 1964 doctoral recipients , 

in education. Conclusions of the study include— (1) Production of researchers is , 
high when the institutions have relevant organizational characteristics, when 

the organizations have a systematic apprenticeship program, and when research ^ 

organizations provide a high index of school services, (2) There is a need for ( 
research organizations to develop their own arrangements for research activity 
and training, (3) Individuals who spend at least six years in teaching are not 
potential research recruits, (4) Doctoral recipients tend to undertake research 
activites during the first year following degree receipt, (5) There is a need 
for improved recruitment procedures, and (6) Education research is, in fact, an 
acadtwic pursuit. 
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Development of an Apprenticeship Work-Study Program Model for the Cross-Training of 

Physical Scientists for Work in Educational Research and Development (Final Report) 
Millstein, Eugene J., Melnotte, Judith M. Dunn, James A, 
P. 0, Box 1113, Palo Alto, California 94302 
American Institutes for Research 

j:£bruary., 1972 , QEG-0-71- 4 712, 106 p , ^ 

♦Research Apprenticeships/*Work Experience Programs/*Educational Researchers/ 
(Bibliographies), (Curriculum Guides), Transfer of Training, Aerospace Industry, 
Research Skills, Career Change/ 



A pilot project to crosstrain unemployed aerospace personnel in educational 
research and development is described. Six persons with bachelor's degrees and 
aerospace experience participated in a thirteen-week training program and received 
stipends. Each trainee was helped in specifying individual learning objectives. 
Learning activities, such as, symposia, group events, guided independent study, 
and apprenticeship tasks, were then prescribed. Half of each trainee's time involved 
work on an actual research project. Job seeking was also included as part of the 
program; three of the six trainees found positions in educational research and 
development. 

An objectives checklist, group activities schedule, individual activities 
logs, and a bibliography of principal assigned readings are presented in the 
report. The administration of the program is described in detail. An evaluation 
of the project, which indicates that the training model proved to be a feasible 
one, is reported in another document, A Study of the Operational Feasibility of 
the Apprenticeship Work-Study Program for Multi-Site Tryout and Evaluation. (AS) 
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The Quality of Research on Education: An Empirical Study of Researchers and 
Their Work. Final Report. 
Persell , Caroline Hodges 
New York, Hew York 

Columbia University* Bureau of Applied Social Research 

May, 1971. 422 p .. OEG -08-080884-45q7 (0 10) _ „ 

♦Behavioral Sciences/*Oata Analysis/*Educdti6naT~Pesearch7^search Methodology 



This study measured and explained the quality of recent research on education. ^ 
A total of 1100 articles and research papers published in 1967-68 were used as ' 
a data source and were qualitatively analyzed with respect to their substance ' 
and methodology. A stratified random sample of 390 papers was evaluated by 
39 judges for contribution to theory, contribution to educational practice, ' 
and use to research methods. A 12-page questionnaire was mailed to all the i 
authors to collect data on their background, training, career lines, attitudes, i 
research context, and topic selection. All the data was analyzed using multi- ^ 
variate tubular analysis, and authors trained in schools of education were compared ^ 
with those trained in the behavioral sciences. Professional socialization was , 
found to be strongly related to research quality, and behavioral scientists did 
better research than education doctorates. The positive elationship between ' 
research socialization and quality was contingent upon certain background I 
characteristics, with female, areligious, and Jewish subjects doing better 
research than males and those belonging to other religious groups, and young 
researchers doing better than older ones. Other related factors were in- 
stitutional setting and being a staff member of a university research center. ' 
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The EducatiOPdl Research Involvement end Capabilities of Institutions for 
Reacher Education, Final Report. 
Puffer, Richard J. 
Evanston, Illinois 
Northwestern University 

June. 1967, 21 8 p>, (Repo rt No> ER-6-2032), 0EC-3>7-062032>0251 

♦College Facul ty/*Curricu1um Research/*EducatronaT Research, Federal Aid, 
Financial Needs/*Learning Processes, Questionnaires, Research Apprenticeships, 
Research Opportunities, Research Problems, Surveys/*Teacher Education/*Teachers 
Colleges/ 

This survey of the 727 members ( f the American Association of Colleges for 
Teacher Education (AACTE) determined their involvement in and capabilities for 
educational research. Usable replies received from 303 persons were coded 
and analyzed. Universities were most heavily involved in educational research, 
usually committing double or triple the resources c(xmiitted by liberal arts 
colleges, teachers colleges, or ''other'* institutions. All institutional categories 
were found to have convertible resources of faculty, space and equipment which 
could be invested in research. Smaller institutions gave evidence of potential 
for significant research if given developmental and financial assistance. Research 
was most often blocked by limitations of faculty time, with money shortages listed 
second. The Federal Government has been the chief cource of research funds, and 
of future research funds. Research is a primary function for a few faculty 
members, but a limited number will be hired primarily for research in 1967-68. 
About one-tenth of the respondents reported training programs for educational 
researchers, but fewer than one-fifth of those not having such programs plan them 
in the near future. Teacher Education, learning processes, and curriculum 
studies in that order are the top priority areas for research. A national 
program for researcher training was recommendeo, along with greater interdisci- 
plinary cooperation and more comprehensive financing for Educational Research at 
all school levels. 
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The Organization of Educational Research in the UnitecFTtates ' 

Sieber, Same D., Lazarsfeld, Paul F. ' 

New York, New York i 

Columbia University, Bureau of Applied Social Research i 

1966, 462 p., (BR-5-0225) i 

*Educational Research, Research, Education/*Organizdtion/*Graduate Study, Graduate ^ 

Professors, Higher Education, Professional Education, Universities, Graduate ^ 

Surveys, Surveys/*Careers , Career Choice, Career Opportunities/*Adm7nistration, I 

College Administration, Information Dissemination, New York City, New York i 

I 

The chief technical problem of this study was to measure the numerous social i 

conditions which might conceivably impinge on the production of research and , 
of researchers by graduate schools of education. The techniques employed included 

(1) Questionnaire surveys of education deans, research coordinators, directors of ! 

research units, project directors in units, and authors of published research ' 

reports, (2) Field interviews and observations of selected research bureaus ! 

and centers and of the work of professional associations, (3) Documentary analysis I 

of school of education catalogs, research articles published in 1964, and research \ 

proposals submitted to the cooperative research program, U.S. Office of Education, , 

and (5) Secondary analysis of survey data from related studies. Numerous con- | 
elusions, recommendations, and tables are prresented in each of eight chapters. 

I 
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First Year Cormnunication Skills Program, Developed by Southwest Regional ' 
Laboratory for Educatianal Research and Development ' 
Jung, Steven, M., Crawford, Jack, J., Kratochvil, Daniel, W. * 
P.O. Box 1113, Palo Alto, Calif. 94302 , i 
American Institutes for Research \ 

September, 197K 0EC>0-70-4892, 55p. | 

♦Curriculum Deve1opment/*Regiona1 Laboratories/*Materia1 Deve1opment/*Reading Materials/ 
*Phonics/*Reading Instruction/*Kindergarten/*Formative Evaluation/*Eva1uation Methodi/ 
♦Positive Reinforcement/*Teacher Behavior/ I 

AIR Product Development Report 1 describes the development of the First Year | 
Communication Skills Program (FYSCP.) FYSCP introduces kindergarteners to reading. . 
It is suitable for children of all backgrounds who meet entry criteria. (90% do.) 
FYSCP includes classroom materials and teacher training materials. The classroom I 
materials for students consist of 52 storybooks for each child, plus practice and ' 
criterion exercises and comprehension sheets. The classroom materials for teachers I 
consist of teaching directions, class records, and alphabet, flash, and animal cards I 
plus badges with which to reward students. The teacher training materials consist of 
fi'Ims, filmstrips, audiotapes, and a lesson plan. The classroom program fills one , 
school year. It does not require special equipment, but does require teachers to | 
follow closely the explicit directions for reinforcing success. The teacher training 
program takes less than two hours. ^ 

The development of the Southwest Regional Laboratory for Educational Research ( 
and Development and its first product, FYSCP, is described. Six organizational . 
charts are presented. The pragmatic philosophy behind the product is explained in 
terms of 13 decisions about the most useful effort the Laboratory could make. An 
appendix presents "Illustrative rules for Developing Instructional Products." Four » 
cycles of formative evaluation are described. A comparison with the Harper & Row 
Basal Reading Program is reported in which FYSCP children did twice as well on J 
word-attack. FYSCP is to be part of a comprehensive elementary English curriculum. ^ 
Seven references are listed. (AS) ^ , 
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Hdwan tngiisn Program, ueveioped by the Hawan btate uepartment 6T 

Education and The University of Hawaii 

Kratochvil, Daniel, W., Crawford, Jack, J. 

P.O. Box 1113, Palo Alto, Calif. 94302 . i 

American Institutes for Research * i 

November, 1971, OEC-0-70-4892, 76 p. . „ ^ 

*Curr7culum Developrnent/*English Instruction/*Disadvantated Youths/*Commum ty Influeftce^ 

♦Multimedia Instruction/*Forfndtive Eva1uation/*Peer Teaching/*Discovery Learning/ i 

*Inservice Teacher Education/*Instructiona1 Systems/*Eva1uation Methods/ I 



AIR Product Development Report z describes the Hawaii tngnsn Program (hlh) 



J. 



I 



and its development. HEP is a complete English curriculum for grades K-12, although 
grades 7-12 are still under development. It Is suitable for all students not needing 
special education. The modular program covers language skills, language systems, and 
literature sub-programs, each in active and passive, oral and written modes. It is ' 
outlined in detail. Materials include texts, workbooks, audiovisual aids, displays,! 
games, and tests. Classrooms need appropriate equipment, adequate wiring, and plenty 
of space for students to move around, as the program emphasizes inquiry approaches, | 
activity-centered learning, peer teaching, and individualization. Supervisory personnel 
attend institutes totalling 9 weeks, then train teachers in workshops totalling 104 ^ouri 
There is an extensive parent-community involvement program. 

The growth of public demand for a better English curriculum is chronicled. The i 
early co-operation of the Hawaii State Department of Education and the University of* 
Hawaii is described. Their separation is identified as one of ten critical decisions 
in the development of HEP. There is a flow chart of major events. An eleven-page * 
appendix summarizes selected HEP formative evaluation studies. 18 references are cited, 
including works by 11 theorists important for HEP. Four studies comparing HEP with the 
standard curriculum are summarized. Students of low socioeconomic status and low to' 
medium scholastic aptitude may do better with HEP. The program h^s been widely ' 
praised. The state has officially adopted HEP and plans to complete its introductioh 
by 1976. (AS) ^ \ 
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Intermediate Science Cuv^riculum Study, Developed by the Florida State ^ 

State University, Intermediate Science Curriculum Study Project ^ 
Kratochvil, Daniel, W., Crawford, Jack, J. ' 
P.O, Box 1113, Palo Alto, Calif. 94302 . ' 
American Institutes for Research ' 

Noupmhpr^ 1Q71 ^ nFr,. n,7n^dflQ9 ^ ii'> p « 

♦Computer Oriented Programs/*Evaluation Methods/*Science Instruction/*Grades 7-9/ 
♦Curriculum Development/*Behavioral Objectives/*Systems Approach/*Diffusion/ 
♦Individualized Curriculum/*Science Materia1s/*Teacher Developed Materials/ ' 

! 

. ^ 1_ 

AIR Product Report 3 describes the development of the Intermediate Science \ 
Curriculum Study (ISCS). ISCS teaches science to junior high school students* | 
It consists of a textbook, workbook, teacher's manual, and laboratory kit, which , 
contains everything needed for experiments except sink, electrical outlets, and | 
tables. Although teacher training is not essential, summer institutes and in- 
service classes are to be offered. The ISCS Project believes that attitudes toward ^ 
science are formed in junior high school. ISCS introduces the student to the method' 
and content of science by gradually increasing the abstractness of concepts presented 
and the student's independence in experimentation. A diagram of the program's content 
flow and facsimile page of the text are reproduced, { 

The development of the program is described and a flow chart of major events 
presented. Eight critical decisions are isolated, including the decision not to 
specify objectives first, but to let them emerge from materials, and also the ( 
decision to compromise whenever necessary for workability in the typical school. , 
Six rounds of formative field trials in regular classrooms and via computer terminal^ 
are summarized. They show improved learning after each revision. The successful 
plan to disseminate ISCS is described. Continued growth is predicted because of the| 
good dissemination plan and the curriculum's uniqueness. (AS) J 
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Science Curriculum Improvement Study, Developed by the Science Curriculum 

Improvement Stud;/ Project , University of California, Berkeley 
Kratochvil, Daniel, W,, Crawford, Jack, J. 
P.O. Box 1113, Palo Alto, Cc^lif. 94302 . 
American Institutes for Research 

November > 1971 > QEC-0^7n^4892. 44 p. 

*Curriculum Development/*Science Instruction/*Kindergarten/*Grades l-6/*Discovery 
Learning/*Teacher Role/*Inservice Teacher Education/*Science Material s/*Teacher 
Programs/*Evaluation Methods/ 

AIR Product Report 4 describes the development of the Science Curriculum 
Improvement Study (SCIS). Designed for all students in grades K-7, SCIS introduces 
basic scientific concepts through guided physical and life science experiments. 
Materials include teacher^s guides, lab manuals, and kits containing everything 
needed except live specimens, which must be ordered separately, standard classroom 
supplies, an electrical outlet, and a sink. SCIS is based on an exploration- 
invention-discovery learning cycle. The organizatirn of training to inculcate 
the required new teaching role is described. The commercial publisner plans to 
continue training if profits permit. 

The gestation and development of SCIS as the brainchild of Dr. Robert Karplus 
are described. A flow chart of major events and a chronology of units published 
are presented. Nine critical decisions are isolated, including the decision to 
require classroom experience of developers. Five small evaluative studies of SCIS 
student attitudes are summarized. It is concluded that continued growth of 
SCIS use will depend on the size of elementary school science budgets and the 
availability of teacher training. (AS) 
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The Sullivan Reading Program, Developed by Sullivan Associates, Menlo 

Park, California 
Thompson, Lorna, J. 

P.O. Box 1113, Palo Alto, Calif. 94302 , 
American Institutes for Research 

November. 197K OEC-0-70-4892. 64 p. 

*Reading Ins true tion/*Kindergarten/*Grades l-3/*Programmed Material s/*Programmed 
Instruction/*Operant Conditioning/*Teaching Machines/*Curriculum Development/ 
♦Material Development/*Applied Linguistics/*Formative Evaluation/ 



AIR Product Development Report 5 describes the Sullivan Reading Program, which 
can be used for teaching reading in kindergarten-grade 3 and also for remedial 
reading instruction. The Sullivan Program consists of programmed readers following 
linguistic principles, instructional materials, testSj and teacher's manuals. 
Neither teacher training nor community involvement is necessary, but two- thirds 
of sales are under "Project READ," wherein Behavioral Research Laboratories, which 
markets most Sullivan products, provides Sullivan reading materials, educational 
consultants, and a parent information program. The Sullivan Program emphasizes 
pupil success and enjoyment. Early versions were tried out by hundreds of children 
and rewritten many times on the basis of their reactions. 

The story of how Dr. M. W. Sullivan created the Program and how it was funded, 
produced, and marketed is told. A flow chart of major events is presented. Eleven 
critical decisions are isolated. Sullivan's early fanatical devotion to programmed 
learning is shown as the essential factor. The successful experiences of five 
school systems with Sullivan reading materials are detailed. Growing sales are 
considered the final proof of the Program's effectiveness. Continued expansion 
is predicted. There is a selected bibliography by Sullivan's coworker. Dr. Allen 
D. Calvin. (AS) 
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ine creative Learning WUq EtiUCi ZUn rrdQVm, U^veloped 5y tfje 

Creative Learning Group 
Thompson, Lorna, J., Kratochvil, Daniel, W. 
P.O. Box 1113, Palo Alto, Calif. 94302 
American Institutes for Research 

December, 1971, OEC-0-70-4892, 47 p. 

'^Lurriculum uevelopment/^urug Abuse/*Health hducfiiti(in/*Uitfusion/*Consultants/ 
*Program Administration/ 



AIR KroducL bevel opmsttt Kfepon 6 describes a dv^ug education py^ogv^am pv^oduc6d 

by the Creative Learning Group. The program presents facts on legal and illegal 
drugs to students in grades K-9 in the belief that informed young people will 
avoid drug abuse. Surburban and inner-city schools use the program. Police . 
departments, libraries, and others have also purchjased it. The materials include 
broklets, 12 Scientific America n reprints listed in an appendix, taped interviews, 
visual aids, stories, questions for discussion, and a teacher's manual. They 
emphasize authenticity and student participation. The program is designed to 
require no expertise of the teacher, but the Creative Learning Group offers 
lectures and workshops to help teachers feel secure with the subject. Parents 
and community may be involved as information sources. 

The case reports how Lawrence McKinney and his associates developed the 
program on a shoestring budget. A flow chart of major events is presented. 
Appendices show sales, resources, and the corporation's income statement. Eleven 
critical decisions are isolated, including decisions not to seek government 
funding and to speed the program to the market as fast as possible. The developers 
relied largely on their own subject expertise, but nine recognized drug authorities, 
whose abbreviated resumes are given, acted as consultants. Time and funds permitted 
little evaluation. Creative Learning Group expects growing sales to finance develop 
ment of curricula in other subjects. (AS) 
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The Frostig Program for Perceptual -Motor Development, Developed by the 

Marianne Frostig Center of Educational Therapy 
Kratochvil , Daniel , W. 
P.O. Box 1113, Palo Alto, Calif. 94302 
American Institutes for Research 

December, 1971, QEC-n-7n>4Rqp , p , 

*Test Construction, Diagnostic Tests, Perception Tests/*Curricu1um Development/ 
*Norms/*Teacher Education/*Curricu1um Guides/*Minima1 1y Brain Injured/*Chi1d 
Deve1opment/*Sensory Training/*Perceptua1 Motor Learning, Perceptually Handicapped, 
Perceptual Development/ 



AIR Product Development Report 7 describes the creation of the Frostig Develop- i 

ment Test of Visual Perception and the Frostig Program for the Development of I 

Visual Perception. The group test screens both normal and educationally handicapped , 

students in grades K-3 for perceptual -motor problems. The program integrates and i 
enhances development of sensory-motor, language, and perceptual skills. It may 
be used as a readiness or corrective program for normal children or as a remedial 

program for handicapped children. The test kit is available to licensed psycholo- ' 

gists and trained teachers. The Frostig Center suggests an 11-point teacher • 

training program and offers demonstrations and a film. The development program i 

is available either as spirit master worksheet sets or as a workbook, both with i 
teacher's guide. 

Dr. Marianne Frostig 's theory of the integrated development of distinct sensory- 
motor, language, and perceptual skills with emotional and social aspects is explainedl 

Abilities covered by the test are outlined. Typical workbook exercises are re- , 
produced. Dr. Frostig's career teaching children with learning disabilities is 

chronicled. The development of the test and workbook materials and the refinement ^ 

of the test and assignment of norms are described. A flow chart of major events , 

is presented. Four critical decisions are isolated. A controlled study of the , 

Frostig program, which showed significantly greater progress by kindergarteners , 

using the program, is described. Eighteen small studies are cited. (AS) i 
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Science - A Process Approach, Developed by the American Association for 

the Advancement of Science 
Sanderson, Barbara, A., Kratcchvil, Daniel, W. 
P.O. Box 1113, Palo Alto, Calif. 94302 . 
American Institutes for Research 

nprpmhpr, 1Q71, OFC^O^l O^aM? ^ Afi p, 

*Curricu1um Deve1opment/*Science Materia1s/*Science Instruction/*Kindergarten/ 
*Grades 1-6/*Behaviora1 Objectives/*Formative Eva1uation/*Eva1uation Methods/ 
*Diffusion/*Program Administration/*Abstraction Levels/ 



AIR Product Report 8 describes the development of the Science--A Process Approach 
program. Sc:ience--A Process Approach teaches students in grades K-6 the intellectual 
processes of science through guided experimentation. It also teaches necessary 
mathematics. Science--A Process Approach comprises 20-25 sequential exercises in 
each of 7 parts. For each exercise, there is a complete lesson plan including 
behavioral objectives and competency measurement activities. For each part there 
is a kit of equipment and supplies necessary for experiments. There is no written 
material for students. An administrator's guide, a teacher training curriculum 
guide, and a teacher training text with subject background reprints are available. 

The curriculum's development by five interdisciplinary summer conferences with 
formative evaluation at 16 schools is described. The schools are listed and the 
six-page teacher feedback form is reproduced in appendices. A flowchart of major 
events is presented. Seven critical decisions, including the decision to base 
the program on Robert Gagne's hierarchy of intellectual skills, are isolated. The 
validation of the hierarchy is described. An independent evaluation of the program 
is reported. Student learning and teacher satisfaction were good. It is feared that 
cheaper, non-AAAS kits may reduce sales. (AS) 
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Variable Modular Scheduling Via Computer, Developed by Stanford 

University and Educational Coordinates, Inc. 
Kratochvil, Daniel, W. 
P- 0. Box 1113, Palo Alto, Calif. 94302 , 
American Institutes for Research 

December, 1971, OEC -0-70-4892, 91 p. 

^Computer Oriented Programs/*Schedule Modules, Flexible Scheduling/*Educational 
Research/*Educational Innovation/*Programniing/*Staff Orientation/ 



AIR Product Development Report 9 describes Variable Modular Scheduling 
Via Computer plan. Variable Modular Scheduling permits schools grades K-12 
to optimize use of all resources by flexible scheduling. The plan includes 
computer programs, instructions and forms for the school to submit data, and I 
consultant services- A diagram of the process of building a master schedule ' 
is reproduced. Since the plan results in some unscheduled student time, staff ! 
parent, and community orientations are recommended. I 

The choice of computer strategy and the development of the plan are described. i 
Modifications to deal with user problems are explained, A flow chart of major 
events is presented. Seven critical decisions are isolated. The stories of 
successful implementation by two schools are told in an appendix. Variable 
Modular Scheduling makes better education possible but, of itself, guarantees 
no improvement. Promotion therefore appeals to those generally Interested 
in educational innovation. Reduced budgets and public reaction against 
experimentation may limit the plan's spread. (AS) 
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sesame street, Developed by tmidrerrs leievision worxsnop 

Kratochvil , Daniel , W. 

P^O. Box 1113, Palo Alto, California 94302 
American Institutes for Research 

December, 1971. 0EC-0-70>4892, 45 p. 

*Preschool Education/*Television , Programing (iJroadcast)/*uisadvantaged Youth/ 
♦Volunteer Training, Tutoring/*High School Students/*Educationa1 Research/ 
♦Evaluation Methods/*Community Invo1vement/*Measurement Techniques/ 



AIR Product Keport lo describes the development or sesame street by tne 
Children's Television Workshop. Sesame Street is a slick, fast-paced, daily, 
one-hour television program designed to enhance the cognitive skills of all 
children aged 3-5, including the disadvantaged. Content includes letters, 
numbers, and geometric forms; reasoning; and social and natural environment. 
Children may watch at home, at viewing centers with trained volunteers, or at 
preschool. A viewing guide and other supplementary materials are available. 
A typical segment of Sesame Street is reproduced. The development of the 
program is described, and a flow chart of major events is presented. The 
process of formative evaluation, unique among television programs, is diagrammed. 
The publicity and community involvement campaigns, including a summer tutoring 
plan employing high schoolers, which have brought the program 90-95% of its 
intended audience, are described. Nine critical developmental decisions are 
isolated. An overall evaluation based on a field test with 943 children is 
summarized. Sesame Street was judged of significant educational benefit, 
especially to 3- and 4-year olds. A small study highlighting the importance 
of emotional and social development that television cannot foster is summarized. 
The honors conferred on Sesame Street are listed. New educational programs and 
other Improvements in children's television as a result of Sesame Street 
are predicted. (AS) 
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Arithmetic Proficiency Training Program, Developed by^Science Research 

Associates, Inc. 
Kratochvil, Daniel, W. 
P.O. Box 113, Palo Alto, Calif. 94302 
American Institutes for Research 

January, 1972. OEC-Q-70-^892, 44 p. 

♦Computer Assisted Instruction/*Arithmetic , Grades 1-8, Remedial Arithmetic/ 
♦Disadvantaged Youth/*Programed Instruction, Programing, Input Output Devices/ 
♦Educational Research/*Sequential Learning/*Mathematic Instruction/ 



AIR Product Development Report 11 describes the Arithmetic Proficiency Training 
Program (APTP). APTP drills students in basic conputational skills via computer. 
It is designed for grades 1-8, but requires approximately grade 4 reading skills. 
APTP covers 29 skills, such as "addition of whole numbers," each divided into a 
hierarchy of subskills. (A skill map is reproduced.) The computer determines the 
student's existing skill level, then leads him up the hierarchy from there, permit- 
ting as much practice as he needs, until he masters the skill as to accuracy and 
speed. APTP includes computer programs with supporting manuals, a teacher^s 
guide, a proctor's guide, and a student logbook for recording progress* It requires 
a timeshared IBM/360 Model 30 computer, a room with teletype-writer computer 
terminals, and a paraprofessional proctor. 

The development of APTP with close co-operation between educational and technical 
staff is chronicled. A flow chart of major events is presented. Four critical 
decisions are described in detail, A formative evaluation study using inner-city 
students in grades 6-8 is summarized. Student interest and progress were good; 
parent and teacher reactions favorable. Several improvements were made based on the 
trial. A small summatfve evaluation study is reported, in which APTP students did 
significantly better than the control group. The scarcity and high cost of computer 
access are judged special, serious obstacles to widespread use of APTP. (AS) 
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The Edison Responsive Environment Learning System, or the Talking Typewriter, 

Developed by Thomas A. Edison Laboratory, A Subsidiary of McGraw Edison Company 
Sanderson^ Barbara, A., I'ratochvil Daniel; W. 
P. 0. Box 1113, Palo Alto, California 94002 
American Institutes for Research 

January. 1972. OEC-0-70-4892. 29 p. 

♦Computer Assisted Instruction/*Reading Instruction/*Programmed Tutoring/*Teaching 
Machines/*Multimedia Instruction/*Special Education/*Audoinstructional Aids/ 
*Typewr i ti ng/*Mater i al Devel opment/ 

AIR Product Report 12 describes the development of the Talking Typewriter. 
The Talking Typewriter, as marketed by the Responsive Environment Corporation, 
teaches reading by a special presentation of the Sullivan program. It consists 
of an electric typewriter, loudspeaker, and display screen, all controlled by a 
built-in computer. (A sketch is reproduced.) The machine displays and pronounces 
program materials, while the keyboard locks so that the student can make only 
correct responses. It requires an air-conditioned room, a teacher for supplementary 
activities, and a paraprofessional machine attendant. The Talking Typewriter is 
suitable for all students who need beginning reading instruction, especially 
disadvantaged, retarded, and handicapped children and illiterate or non-English 
speaking adults. 

Early research using the electric trypewriter as an instructional devise is 
summarized. The development of the Talking Typewriter and Sullivan software is 
described. A flow chart of major events is presented. Four critical decisions are 
described in detail, and the lack of systematic formative evaluation is discussed. 
Results of two large Talking Typewriter programs are summarized. Students made 
significant gains, but how much wac due to the program was not clear. High cost 
is judged the principal obstacle to widespred use of the Talking Typewriter. (AS) 
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Holt Social Studies Curriculum, Developed by Carnegie Social Studies 

Curriculum Development Center, Carnegie-Mellon University 
Sanderson, Barbara, A., Kratochvil, Daniel, W. 
P. 0. Box 1113, Palo Alto,, California 94302 
American Institutes for Research 
January, 1972, 0EC>0-70-4892, 61 p. 



♦Curriculum Development/*Social Studies, Grades 9-12/*Test Construction/*Educational 
Researchers/*Qual ifications/*Program Administration/*Inquiry Training/*Professional ' 
Education/ I 



AIR Product Development Report 13 describes the Holt Social Studies Curriculum 
and its precursor, the Carnegie Social Studies Curriculum. The curriculum introduces 
students in grades 9-12 to the methods of social and behavioral sciences and to key 
concepts in several disciplines. Teaching subject matter and developing attitudes 
are secondary goals. The curriculum has been adapted for average and better students' 
from one designed for the top 20% only. Materials for each of seven courses consist df 
a student textbook of readings, an audiovisual kit, a test booklet, and a tearher's I 
guide with daily lesson plans, which emphasize guided discussion. Teacher training i 
is not necessary, but films, books, and assistance of salesmen are available. , 
The content of the 7 courses is summarized. The materials are described. Two lesson, 
plans are reproduced in an appendix. 

The development of the curriculum and its trial in Pittsburgh schools are described, 
A flow chart of major events is presented. Seven critical decisions are isolated. | 
Some things the development project staff learned from experience are extracted from 
the project's final report--above all to budget sufficient time and money. A 
doctoral program in curriculum development that has grown out of the project is 
described. Two evaluations of the precursor curriculum for able students which used 
standardized tests and one which used a specially constructed instrument are 
surmarized. The standardized tests showed little difference between study and control 
groups » but the special instrument showed the study group significantly better in 
social studies inquiry skills. Continued widespread use and imitation are forecast. 
(AS) 
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Distar Instructional System, Developed by Siegfried Engelmann and 

Associates, The University of Oregon * 

Kin, Yungho, Berger, Bonita, J., Kratochvil, Daniel, W. | 

P.O. Box 1113, Palo Alto, Calif. 94302 * ^ 

/^jnerican institutes for Research ^ 

.lanuary, Ofr^n^lO^/LMO^ p [. 

*Reading Instruction/*Mathematics Instruction/*Kindergarten, Grades l-3/*Curriculum | 

Development/*Disadvantaged Youth/*Preschool Programs/*Language Arts/*Behavioral . 

Objectives/ ' 

I 

" L 

AIR Product Report 14 describes the development of the Distar Instructional ^ 

System. Distar teaches the basic concepts and skills of reading, language, and ! 

arithmetic needed for succce'^<^ in school. It is designed for preschool through j 

primary grade children, especially the disadvantaged. The materials consist of | 

teacher's kits with verbatim lesson plans anrl visual aids, and student workbooks , 
and take-home worksheets. Distar is based on the theory that IQ indicates what 
a child has been taught, not wha he is able to learn. It anphasizes systematic, 

small-group drill on single, sequenced concepts. Teacher training manuals, research ' 

data sheets, and three books by Distar *s developers are available. The publishers » 

provide a free, two-day orientation workshop to new teachers. The curriculum is ^ 

outlined in detail. Its development is described. A flow chart of major events is | 
presented. Seven critical decisions are isolated. Four evalutions of precursor 

programs are summarized. Two studies by the developers and one by Head Start 1 

showed significant, sustained IQ increases in disadvantaged children. A study by i 

Ypsilanti school district comparing three preschool curricula for disadvantaged i 
children showed equally good results with all three. Three evaluations of the 

Distar program in schools with many disadvantaged students are also summarized. \ 

Results were fair good with Distar alone, excellent when individualized instruc- , 

tion was added. About 3,000 schools serving the disadvantaged use Distar— in- i 

eluding 20 Follow Through sites. Its spread is judged dependent on the success of i 

teacher training and of ongo^^g materials revision. (AS) i 
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Materials and Activities for Teachers and Children - The Match Program 

Developed by the Children's Museum, Boston, Massachusetts 
Sanderson, Barbara, A., Kratochvil, Daniel, W. 
P.O. Box 1113, Palo Alto, Calif. 94302 
American Institutes for Research 
January. 1972. 0EC>0>70>4892. 54 p. 



*Instructional Materials/*Material Development/*Learning Modalities/*Soc1al 
Studies/*Kindergarten, Grades l-6/*Museums/*Activity Units/ 



AIR Product Development Report 15 describes the Materials and Activities for 
Teachers and Children (MATCH) boxes and their development. The MATCH boxes 
stimulate development of subject knowledge, critical thinking, problem-solving 
skills, and self-awareness. They are designed for grades K-6. The 16 boxes deal 
with science? social studies, and language. They are for rent from the Children's 
Museum, Boston. The three for sale from the American Science and Engineering Co., 
Boston, all deal with social studies. Each box contains real objects, reproductions, 
audiovisual aids, and supplies for diverse nonverbal activities on some theme (such 
as Japanese family life) plus a teacher's guide. The contents and use of the 16 
boxes are described, with frank comments from the development project's final report. 
The development of the boxes is chronicled. Charts of the schedule and a flow 
chart of major events are presented. Seven critical decisions are isolated, 
including the decision not to consider commercial production and marketing problems 
during development. The developers observed classroom use of materials during the 
creation of each box. The final field test of each box consisted of trial and 
observation in 15-22 classrooms, plus teacher questionnaires. Student interest 
and learning were judged high. Lack of money is named as the only barrier to wider 
use of MATCH boxes. (AS) 
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Developmental Economic Education Program (DEEP), Developed by the Joint Council on 

Economic Ed., in Cooperation with its Affiliated State Councils & Centers 
Kim, Yungho, Kratochvil, Daniel W. 
P, 0. Box 1113, Palo Alto, California 94302 
American Institutes for Research 

January „ 1Q7?, ()fC^()^7n^aRQ9, A7 p. 

*Curriculum Development/*Economic Education/*Kindergarten, Grades l"12/*Fducational 
Innovation/*Community Cooperation/*Inservice Teacher Education/*Teacher Developed 
Materials/ 



AIR Product Development Report 16 describes the Developmental Economic Education 
Program (DEEP) and its creation. DEEP helps elementary and secondary schools, other \ 
than those for exceptional children, to improve curricula for basic economic \ 
education. It emphasizes teacher education. The program offers a choice of i 
products and services for schools to adapt. The present report lists DEEP's 
published materials; they include Handbook for Curriculum Change , course outlines, ' 
bibliographies, filmstrips, etc. The Joint Council on Economic Education has ' 
produced the Test of Economic Understanding and a televised college course, "The 
American Economy, useful in the program. Consulting services are provided through i 
local Council affiliates. 

The careers of key personnel on the DEEP project are summarized. The development 
of DEEP by the Council, with the participation of 30 named school systems, is 
described. Flow charts show the program *s structure in a typical school system I 
and community, the school's activities in implementing DEEP, and major events in 
the Council's developmental effort. Ten critical decisions are isolated. The , 
Council's criteria for publication of DEEP materials produced by the schools are 
stated. Ten conclusions of the Psychological Corporation's overall evaluation , 
of the program are quoted. Three controlled studies of DEEP's use in the classroom , 
are summarized. Results are ambiguous. The program is judged helpful to schools , 
wishing to improve curricula. (AS) i 
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Individually Prescribed Instruction--Matheniatics nPI--Math), Developed by the ' 

Learning Research & Development Center, etc. 
Jung, Steven, M. 

P.O. Box 1113, Palo Alto, Calif. 94302 
American Institutes for Research 

January, 1972, OEC-0-70-4892, 44 P> _ ^ 

*Teacher Aides/*Peer Teaching/*Behavioral Objectives/*Mathematics Instruction/ 
*Kindergarten, Grades l-6/*Programmed Instruction/*Evaluation/*Regional Laboratories/ 
*Diffusion/*Individualized Instruction/ 



AIR Product Report 17 describes the development of Individually Prescribed \ 

Instruction in Mathematics (IPI Math). IPI Math teaches arithmetic to students , 

in grades K-6, including the disadvantaged and retarded. IPI Math consists of ' 

classroom and teacher training materials. Classroom materials include placement I 

tests, pre- and post-unit tests, and curriculum-embedded tests; semi -programmed ' 

work booklets; audiovisual aids; and forms on which the teacher keeps track of » 

student progress. Teacher training materials include a six-volume guiHe. The i 

developers show principals how to use the guide for inservice training. They i 

also offer a summer workshop. IPI Math is based on a hierarchy of 363 student , 

performance objectives. The teacher monitors each student's progress and j 
prescribes the next step individually for each. Two paraprofessional aides per 

class are needed. The teacher's role in IPI is also described. ^ 

Organization charts of the Learning Research and Development Center and of [ 
Research for Better Schools (the Middle Atlantic Regional Laboratory) are 

presented. The development of IPI is described and a flow chart of major events ' 
presented. Tasks needed for five major project activities are listed in an 

appendix. Eleven critical decisions are isolated. Overwhelming early interest ' 
required criteria for selecting pilot schools and assessing the fidelity of 
implementation at each; the criteria are reproduced. The philosophy and methodology ' 

of IPTs ongoing formative evaluation are discussed. General results of over 25 J 

comparative summative evaluations of IPI are reported.- Student learning and j 

attitudes were improved and parental reaction was favorable. (AS) ^ 

i 

\ 

I 
I 
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I 
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The Cluster Concept Program Developed by the University of Maryland, 

Industrial Education Department 
Kratochvil, Daniel W., Thompson, Lorna, J. 
P.O. Box 1113, Palo Alto, Calif. 94302 
American Institutes for Research 
January. 1972. nEC-n^7n^4fiQ? , fi.-^ 



*Behavioral Objectives/*Vocat1onal Education/*Grades n-12/*Sk111s Development/ 
*Occupational Clusters/*Curriculum Development/*Inservice Teacher Education/ 
*Occupational Guidance/ ' 



AIR Product Report 18 describes the development of the Cluster Concept Program, i 
The program prepares students for entry to occupational clusters in construction, i 
electro-mechanical installation and repair, and metal forming and fabrication. , 
It is designed for lith and 12th grade students who have not chosen specific i 
occupations and do not wish to rule out college. The program consists of three 
course outlines and three sets of lesson plans, which analyze the skills needed ' 
for entry to occupations in each cluster and identify their common learning ' 
objectives, (Examples are reproduced in appendices.) Teachers need skill in ' 
several crafts plus the ability to integrate learning objectives. Equipping « 
workshops costs $25,000-$40,000. I 

The development of the program is described. A flow chart of major events 
is presented. Seven critical decisions are isolated. The selection and prepara- 
tion of field test teachers are described. Two-year trials in four high schools I 
are summarized. Students in some Cluster Concept Program classes learned signifi- 
cantly more than those in regular vocational classes, but due to late arrival 
of equipment and supplies anH poor teacher preparation, students in some others did 
not. 60% of grdduates were employed at jobs in their cluster within 4 weeks. The 
Cluster Concept is judged worthy of further research. (AS) 
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The Taba Social Studies Curriculum, Developed by the TABA Social ^ 

Studies Curriculum Project, San Francisco State College ' 

Sanderson, Barbara, A., Crawford, Jack, J. ' 

P.O. Box 1113, Palo Alto, Calif. 94302 • ' 

American Institute*; for Research ' 

March, 1Q72. ngC-n^7n-4fiQ2. fi7 p. ' 

♦Social Studies, Grades 1-8/*Curricu1um Deve1opment/*Protoco1 Material s/*Test 
Construction/*Eva1uation/*Inservice Teacher Educatfon/*Teacher Developed Materials/ 

*Tests/*Discussion (Teaching Technique)/*Cognitive Processes/ ' 



AIR Product Report 19 describes the development of the Taba Social Studies i 

Curriculum. The curriculum consists of teaching guides with detailed lesson I 
plans for heterogeneous classes, grades 1-8. It stresses learning to form 

and use generalizations and developing appropriate attitudes. An explanatory 1 
handbook is available. Dr. Hilda Taba's theories, which draw on Plaget, are 

explained with examples from the teaching guides. The development of the ' 

curriculum by teachers and departmental staff of the Contra Costa County Schools, ' 

under Taba's leadership, is described. The project's schedule and g flow chart ' 

of major events are presented. Nine critical decisions are isolated. Development i 

and diffusion plans centered on teacher involvement right from the starts Four i 
patterns of teacher orientation are described. Construction of evaluation instru- 
ments is described. Sample items from three tests are reproduced: the Application 

of Generalizations Test, the Test of Ability to Explain, and the Interpretation i 

of Data Test. Five free-response questions are given. A controlled evaluation , 
of the curriculum is reported. The reasons why this type of evaluation, required 
by the U. S. Office of Education, was not appropriate are presented. A survey 

showing greater interest on the part of Taba students and a study showing their , 

higher level of discussion are surmiarized. The curriculum is used nationally. , 

Meanwhile, Addison-Westley, the- publisher , is revising the teacher guides and , 

preparing student texts for commercial publication. Competition from other i 

process-oriented social studies programs may, it is considered, limit the Taba i 

curriculum's spread. (AS) , 
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Facilitating Inquiry in the Classroom, Developed by the Northwest 

Regional Educational Laboratory 
Crawford, Jack, J. 

P.O, Box 1113, Palo Alto, California 94302 

American Institutes for Research in the Behavioral Sciences 

■March. 1Q72. QEC^Q^7n^48qp. 33 p. \ 

♦Inquiry Training/*Curriculum Development/*Science Instruction/*Inservice Teacher 
Education/*Teacher Role/*Qualifications/*Protocol Materials/ 



AIR Product Report 20 describes the development of the Facilitating Inquiry i 
training package. The Facilitating Inquiry training package teaches teachers how to | 
foster self-directed discovery learning. The training package consists of a leader's, 
guide with detailed learning objectives and lesson plans, ditto masters for student i 
materials, four protocol tapes, and some demonstration equipment. The program takes 
42 class hours. It can fit an extension, workshop, or summer school schedule. Any ' 
trainee who has been through the program can lead a group, for it emphasizes role ' 
playing and group evaluation rather than the teacher's imparting knowledge. I 

The development of Facilitating Inquiry under the Northwest Regional Educational i 
Laboratory is chronicled. An organizational chart of the Laboratory and a flow charti 
of major events are presented. Eight critical decisions are isolated. The considerar 
tions preventing development of guru-disciple training network instead of the present 
training package are explained. A field test of the program is described. Learning I 
was adequate, though greater with expert group leaders. High school teachers respondpd 
better than grade school teachers. Reasons suggested for the moderate use of , 
Facilitating Inquiry include its requiring 42 hours, the shortage of qualified group ^ 
leaders, and the plethora of similar curricular materials. (AS) 
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The Mini Course, A Microteaching Approach to Teacher Education 

Borg, Walter, R., Kelley, Marjorle, L., Langer, Philip, Gall, Meredith 
8701 Wilshire Boulevard, Beverly Hills, Ca,lifornia 90211 
Macmillan Educational Services, Inc. 

1970. 2 56d> ^ , 



*Microtedching/*Behavioral Objectives/*Teacher Education/*Videotape Recordings/ 
*Teaching Skil ls/*Educational Research/*Curriculum Development/*Fonnative 
Evaluation/*Tu tori ng/*Mathema tics Instruction/*Language Development/*Educationa1 
Innovation/*Class Management/ 



After an analysis of the reasons why present teacher education programs fail, 
the advantages of the Minicourse instructional model are explained. The Minicourse 
is a short course, suitable for preservice or inservice training, which shows the 
teacher behavlorally defined teaching skills, then lets him practice with a minia- 
ture class, observe himself on videotape, and practice again. The research 
relevant to Minicourse design is reviewed. The Far West Regional Laboratory's 
philosophy and methods of ed'jcational research and development are discussed. The 
rationale, development, and field-testing of Minicourses on the following skills 
are described: leading class discussion, developing language skills of disadvantaged 
children, organizing kindergarteners for independent study, and mathematics tutoring. 
Observation of mathematics instruction is reported » All Minicourses proved effective 
and popular in field tests. Eighteen questions requiring further research are 
posed. Fourteen ongoing pilot studies are named. Two other instructional models 
under development are described: Simulation and Stimulation-Discussion-Action. 
Progress on ten new Minicourses is reported. It is concluded that improvement 
of education will require research that develops operational products. (AS) 
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■I 

Crawford, Jack J., Kratochvil, Daniel W. , and Wright, Calvin E. i 
P. 0. Box 1113, Palo Alto, California , i 
American Institutes for Research in the Behavioral Sciences i 

1972, 200 p. I 

*Report/*ABE/*ABE Program Development/*Instructional Material Evaluation, Level 1-HS' 
(Grades 1-12)/ i 



Approximately 117 educational products and "packages" were selected tor study J" 
in this final report in an effort to develop more precise guidelines for assessing 
potential impact in the classroom situation. The study involved development of * 
initial selection criteria, systematic case studies following the products from | 
origin to diffusion, and preparation of generalizations as to the types of products I 
that would have most impact on the educational process if adopted for usage. The \ 
116 packages of which 21 were studies in depth, cover these content areas: mathe- i 
matics, science, social studies, language arts and vocational education. Textbooks,] 
workbooks, films, non-graded, individulized and traditional materials are covered. ^ 
This study should help educators to pre-assess potential impact in their classrooms j 
of available educational materials. (JR) 

I 
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Installing A Coordinated Information Network in a State Education ^ 

Agency: A Case Study of the Decision Process in New York i 

Hull, William L., Benson, Gregory, Jr. » 

1900 Kenny Road, Columbus, Ohio 43210 I 

The Ohio State University i 

February, 1972> OEG-3-7-0001 58-2037, 96 p. ' 

*taucdtiona1 Cnange/*Educationa1 Innovation/*Information Needs/*Information 
Networks/ ^Organizational Change/*Research Coordinating Units/*State Departments of 

Education/ ' 

I 

, ^ \ L 

This research report describes events and decision processes which encourage , 

or inhibit the installation of an innovation in an educational agency. The 

innovation selected for this case study was the installation of a coordinated 

information network in the New York State Education Department. j 

This study was conducted during the time much of the decision-making occurred. ' 

Contact was made with the director of the New York Research Coordination Unit I 

in February, 1970. The study was completed in September of the same year. I 

Some of the data collected overlapped the study time frame, March, 1968 to j 

September, 1970, but most of the information was obtained ex post facto . , 

The study examines several levels of the information network within the , 
state, e.g., state, regional, and local, in order to describe a complete 

system with feedback from local to state level professionals. I 

The primary thrust of this project was the investigation of the procedures for | 
installing the system in the State Education Department. Hopefully, the delinea- 
tion and analysis of such procedures in reference to change process concepts have ' 
yielded some significant findings for anyone attempting to implement a similar ' 
idea at the state level. * 

Chapter One describes the rationale, objectives, and procedures of the study. ' 

Chapter Two provides background information to the reader and sets the stage ' 

for the comments to follow. Chapter Three focuses on the events and processes * 

which occurred within the State Education Department. Organizational boundaries, | 
cormunications, etc., were analyzed as they influenced the decision process. 

Chapter Four summarizes the responses from institutions cooperating in the project ' 

and assesses the condition of information services in the state. Striking < 

differences are apparent in the way classes of institutions responded to i 

information needs of user groups.. Chapter Five lists the findings, implications, , 

and recommendations of the study. i 
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Research on the Use of Educational Products, (1) Marketing Study: 

Management Training Unit, Appendix 
Hutchins, C. L,, Mclntyre, D,, Mitchell, K, Harding, J. 
1 Garden Circle, Hotel Claremont, Berkeley, California 94705 
Far West Laboratory for Educational Research and Development 

July, 1972, 30 p. 

*Audiences/*Diffusion/*Educationa1 Demand/*Formative Evaluat1on/*Educational 
Objectives/*Educationa1 Prob1ems/*Instructiona1 Material s/*Marketing/*Professiona1 
Education/*Use Studies/ 

This study was undertaken to secure Information to assist in planning for the 
dissemination and utilization of the first three Management Training Units, in 
a larger systems development effort by the Far West Laboratory for Educational 
Research and Development, Information focused on two major questions: What 
distribution mode should be selected to reach target audiences? And, what do 
users or potential users perceive as the strengths of the materials which in 
turn would provide direction in planning publicity for the units? 

The study was conducted during December, 1971, and January, 1972* with 70 
subjects. In the first part of the study, 31 subjects who has used one 
of the training units in a test situation were interviewed in person or by 
phone. The interviews lasted approximately 30 minutes. Fourteen subjects had 
taken the Problem Analysis Training Unit, 12 had taken the Deriving Curricular 
Objectives Training Unit, and five had taken the Goal Setting Training Unit, 
In the second part of the study, 39 interviews were conducted with individuals 
unfamiliar with the materials but who were potential users of the units. Each 
of these subjects scanned one or more of the training units, following a prescribed 
set of reviewing procedures, and then responded to a brief questionnaire; these 
procedures took about 45 minutes. Twelve subjects examined the Problem Analysis 
Training Unit, 15 examined Deriving Curricular Objectives and 12 examined the 
Goal Setting Unit, In addition to these two sub-studies, a random sample of 
college catalogs were screened to ascertain what percentage of colleges and 
universities would have departments where the units might be used as class 
materials and how many classes might be expected to use the materials. The 
units were well received, both by those who had used them and by those who were 
first introduced to them during our survey. For example, if pricing and 
competitive products are not considered, about 95% of the administrators and 
college professors said they would want a copy for themselves. When they were 
told the units might cost $20 each, 11% of the administrators would still secure 
individual copies but only 50% of the professors would. These and other data lead 
us to conclude that the units are perceived to fill a need and will be secured by 
a significant number of people Improperly priced and distributed. 
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The Oregon Studies in Educational RDD&E. Vol. IV: Profiles of Exemplary Projects in' 

Educational RDDSE (Part One of Three Parts - Research & Development). Final Repont, 
Ammerman, Harry L. (Ed.) and Others » 
Monmouth, Oregon , i 

Teaching Research i 

Grant No. OEG-0- 70^4977, 1972 i 

^Academic Achievement, Action Research/*Case Studies, Consolidated Schools, Early ^ 
Childhood Education, Educational Innovation/*Educational Research/*Evaluation, » 
Learning Activi ties/*Measurement Goals, Microforms, Objectives, Organization/ I 
^Program Evaluation, Reading Research/ , 



This volume contains 20 case study profiles ot educational KUD&E projects and, 
as such, constitutes the data base for the Oregon Studies. This part (Part 1) 
of Volume IV contains profiles of five research and three evaluation projects along 

with information that describes the development of the profiles, explains how to ' 

read the profiles, and includes a glossary of common profile terms. Each profile I 

contains three sets of data: (1) descriptors of general project characteristics, i 

(2) descriptors of personnel working within the projects, and (3) descriptors of i 

the work requirements within a project. The central data reported in a profile | 

deal with project work requirements. In this regard, each profile describes the , 

output of work effort; the standards established for those outputs; the operations ^ 
required to produce outputs to specified standards, and the knowledge, skills, 

and sensitivities needed to carry out those operations. Related documents are i 

EA 004 582-585 and EA 004 587-589. i 
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The Oregon Studies in Educational RDD&E. Vol. IV: Profiles of Exemplary Projects 

in Educational RDD&E. (Part Two of Three Parts - Development) Final Report 
Anmerman, Harry L. (Ed.) & Others 
Monmouth, O»^egon 
Teaching Research 

Grant No. 0EG-0-70>4977, 1972 ^ 

♦Behavioral Object1ves/*Case Studies, Computer Science, Course Object1ves/*Curr1culum 
Development/*Educationa1 Development, Educational Research, Instructional Media, 
Instructional Technology, Leadership Tra1n1ng/*Measurement Goals, Objectives/ 
♦Program Evaluation, Teacher Education, Vocational Development/ 

This volume contains 20 case study profiles of educational RDD&E projects and, as 
such, constitutes the data base for the Oregon Studies. This part (Part 2) of 
Volume IV contains profiles of seven development projects along with information 
that describes the development of the profiles, explains how to read the profiles, 
and includes a glossary of common profile terms. Each profile contains three 
sets of data: (1) descriptors of general project characteristics, (2) descriptors 
, of personnel working within the projects, and (3) descriptors of the work require- 
ments within a project. The central data reported in a profile deal with project 
work requirements. In this regard, each profile describes the output of work effort; 
the standards established for those ouputs; the operations required to produce 
outputs to specified standards; and the knowledges, skills, and sensitivities needed 
to carry out those operations. Related documents are EA 004 582-586 and EA 004 588- 
589. 
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The Oregon Studies in Educational RDD&E. Vol. IV: Profiles of Exemplary Projects 

in Educational RDD&E {Part Three of Three Parts - Diffusion). Final Report 
Ammerman, Harry L. (Ed.) & Others 
Monmouth, Oregon 
Teaching Research 

Grant No, QEG^0>7Q-4977 , 1972 

*Case Studies, Cormnunity Invo1vement/*Diffusion, Disadvantaged Youth, Educational 
Development, Educational Innovation/*Educat1ona1 Research, Educational Television, 
Information Centers, Information Dissemination, Information Retrieval , Measurement 
Goals, Objectives/*Program Evaluation, Reading Programs/*Research Utilization, 

School Visitation/ 

This volume contains 20 case study profiles of educational RDD&E projects and, 
as such, constitutes the data base for the Oregon Studies. This part {Part 3) 
of Volume IV contains profiles of five diffusion projects along with information 
that describes the development of the profiles, explains how to read the profiles, 
and includes a glossary of common profile terms. Each profile contains three sets 
of data: (1) descriptors of general project characteristics {2) descriptors of 
personnel working within the projects, and (3) descriptors of the work requirements 
within a project. The central data reported in a profile deal with project work 
requirements. In this regard, each profile describes the output of work effort; the 
standards established for those outputs; the operations required to produce outputs 
to specified standards; and the knowledges, skills, and sensitivities needed to 
carry out those operations. Related documents are EA 004 582-587 and EA 004 589. 



ERLC 




"improving Urgamzatlona I Processes in Unitized blementary Schools," in The Oregon r 

Studies in Educational RDD&E, Volume IV, Part One i 

Hill, Herbert E. i 

Monmouth, Oregon i 

Teaching Research i 

Project No, 0-0701, Contract No. OEG-0-70-4977, 1972 i 
^Research Skills, Educational Researchers, Research Projects/*Hrogram Administration'/ 

*Team Teaching, Team Training/*Employment Qual if ications/*Professional Education/ i 

*Admini strati ve Change/ I 



Oregon Profile 1 1s a case study or the unitized Kesearcn project, me unitizep 
Project aimed to compare different intensities of staff preparation for team teaching 
by training school staffs and observing the results. The project was part of the ' 
Center for Advanced Study of Educational Administration's Program 30. The Unitized j 
Project drew on and contributed to other Center projects. The staff included a I 
director, co-director, and research associate, who share in decisions, plus nine | 
research assistants. None worked full-time on the Unitized Project, Communications! 
among project staff often took written form, so writing skill was especially | 
important. Most staff held degrees in psychology. The abilities to deal with ^ 
data and with people were both important to the project, but it was hard to find ^ 
research assistants able to deal wqll with data. It is concluded that training 
for natural situation research should emphasize not the controlled procedure i 
possible in a laboratory but flexibility of procedure with clarity of goals* j 
Researchers need to be flexible in outlook, adaptable to irregular work schedules, , 
and sensitive to coworkers' feelings. Classroom study cannot develop these ^ 
qualities; training programs should include work experience. (AS) ' 
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"Consolidation: The Effects of a Modernizing School System on the Aspirations, Achieve- 
ment, & Adjustment of Students in an Appalachian County,*' in Oregon Studies, Vol • IV-1 
Pielstick, C. Dean ! 
Monmouth, Oregon i 
Teaching Research i 

Project No, 0>0701, Contract No, OEG-0-70-4977 , 1972 I 

*Research Skills/*Educational Researchers /^Research Projects /*SocTal Factors/ ^. 
^Consolidated Schools/*Case Records/*Research Problems/*Questionnaires/*Res8arch ' 
Apprenticeships/*Professional Education/ I 

I 

Oregon Profile 2 depicts the Consolidation Project, a research project directed] 
by a sociology professor. The Consolidation Project aimed to discover the effects 
of school consolidation on high school students. The project took advantage of a ' 
natural control group situation created when three rural high schools consolidated ' 
and a fourth did not. Work started before a proposal for funfing was even written, I 
The staff comprised a professor and a research assistant. The work style was I 
egalitarian, the schedule flexible. School records proved unreliable data sources, i 
so the assistance of school personnel was invaluable. Staff needed background | 
information about the local schools and community, general, knowledge of sociology ^ 
and education, and the imagination to discover alternative kinds of evidence about j 
a variable. Guided research in graduate school is suggested as the best training 
for educational research, (AS) ' 
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"An Analysis of the Stablility and Instability of Student Growth," in The Oregon ^ 

Studies in Educational RDD&E, Volume IV, Part One t ' 

Jones, Diane G. \ ' 

Monmouth, Oregon - \ ' 

Teaching Research ; ' 
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*Research Projects/*Research Design/*Research Criteria/*Program Descripti(jns/*Case 
Studies (Education)/*Cross-sectiona1 Studies/*Longitudinal Studies/*Statistical 
Studies/*Academic Achievement/*Achievement Rating/*Academic Performance/*Low ' 
Achieves/*Long Achievement Factors/ ^ 

, ^ L 

The Oregon Profile 3 case study describes a research project on the academic i 
growth of students, grades 5-12, as measured primarily by achievement and ability i 
objective tests. The subject areas tested are science, social studies, mathematics, 
rSw^ding, writing, and listening* Growth in various subject areas is to be judged | 
relative to curriculum, sex, age, race, socio-economic status, region of the U.S., 
high school graduation class size, and city population. Organization structure, ' 
personnel roles and functions and an annotated list of descriptive and interpretive' 
output are discussed as project parameters. The following project dynamics are I 
discussed: management and communication processes, interrelationships among project 
staff and between the project and parent agency, and physical setting. The queried i 
staff listed various prescribed training, work experience, knowledges and skills as. 
implications for training. (AS) 
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"Perceptional and Memory Components in Reading," In The Oregon Studies in Educationa'l 
RDD&E, Volume IV, Part One ' 
Myers, R. E, ' 
Monmouth, Oregon , ' 

Teaching Research » 

Prelect No, 0-070K Contract No, OEG-0-70-4977 , 1972 ^ |_ 

♦Experimental Programs/*Educationdl Experiments/*Research Projects/*Research Criteria/ 
♦Research Design/*Reading Research/*Developmental Reading/*Beginnin9 Reading/*Readirlg 
Skills/*Reading Improvement/*Reading Processes/*Early Reading/*Reading Development/ 1 
*Case Studies (Education)/ i 



Oregon Profile 4 describes a research project which investigates the nature of i 
the reading process in the normal reader, grades 2-6, and initial reading acquisition 
in nursery and kindergarten children from disadvantaged environments- The rationale 
behind the program is that 10-25% of all children have difficulty learning to read, j 
The project staff expects to obtain knowledge from experimental results on how ' 
skilled readers perform and the cognitive skills needed to learn to read; also to ! 
develop testing packages and training techniques to measure and improve reading I 
readiness skills. The project staff, outputs generated, and the dependent relationn 
ships of the outputs are discussed as project parameters. Twenty-eight project j 
outputs are briefly described and tabular data reported from the staff interviews, j 
(AS) 
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"A Research Project to Determine the Student Acceptability & Learning Effectiveness ' 
of Microform Collections in Community Jr. Colleges, Oregon Studies, Vol. IV, Part 1 
Crowhurst, Norman H. ■ 
Monmouth, Oregon I 
Teaching Research i 

Prnjpnt Nn. 0^0701 , Contract No, OEG-0-70-4977 . 1972 ' 

*Microforms/*Library Research /*Juni or College Libraries/*Use Studies/*Institutionfirn 
Research/*Experimenta1 Programs /*Educat"Jona1 Specifications/*Educationa1 Experiments'/ 
♦Program Descriptions/*Research Projects/*Research Design/*Research Criteria/ I . 

I 

Oregon Profile 5 is concerned with an experimental project in junior colleges | 
for determining the quality and degree of acceptability of various microforms as . 
opposed to more traditional written media. The project also proposes to investigate 
the effect of microform utilization on student learning. The rationale behind the 1 
project is that microforms are economical because they are less expensive than hard ' 
copy, use less storage space, and could lead libraries to distribute, rather than I 
circulate materials. The parameters of the project include a discussion of project i 
staff and an index of products and management responsibilities. The study defines | 
and comprehensively details nine major product results and eight management , 
responsibilities. These were obtained from staff interviews. Project dynamics inc1,ude 
discussions of the focus of the project and internal staff, considerations on 
cooperation within the project, and the use of hardware and software. Five necessarty 
training areas are detailed as implications for training. (AS) I 
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'*A Research and Evaluation Unit in a Public School System: The Office of Research &' 
Evaluation of the School District of Philadelphia'; Oregon Studies, Vol. IV, Part 1" 
Smith, Clark A. ' 
Monmouth, Oregon * 
Teaching Research • 

Project No> 0^0701, Contract No. OEG-0-70-4977. 1972 | 

*Research Skills/*Educational Researchers/*Data Process ing/*Academic Records/*Test 
Construction/*Questionndires/*School Statistics/*Computer Oriented Programs/*Researc!h 
Design/*Professional Education/ 1 



Oregon Profile 6 depicts the Office of Research and Evaluation (ORE) of the 
Philadelphia school district. ORE aimed to gather, preserve, and present data 
on Philadelphia's schools useful to decision makers and the general public. It 
designed tests, arranged for their production and administration, stored scores, 
and other statistics in computers, and put the data into usable form* The staff 
comprised an executive director, four divisional directors, supervisors, numerous 
research assistants and interns, and clerks. All non-clerical positions involved 
substantial quasi-administrative work— meeting, advising, etc. Relations between 
the school administration, which ORE served, and teachers and principals, with 
whom ORE worked, were strained. ORE staff needed tact. Position in the office 
hierarchy did not depend on academic degrees, because the office preferred to 
finish training inhouse. Research interns were trained on the job to be research 
assistants. It is concluded that training should foster in the researcher the 
attitude that "those he hopes to serve are rational people," (AS) 
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"The Evaluation of the Early Childhood Education Program,'* in The Oregon Studies in ' 
Educational RDD&E, Volume IV, Part One ' 
Hill, Herbert E. ' 
Monmouth, Oregon - ' 

Teaching Research ' 

-Project No, 0^^0701, Contrftrt Nn, nFG>0^70-4q77, 197? . !_ 

*Experimenta1 Programs/*Educationa1 Experiments/*Research Projects/*Research Design/ 
♦Research Criteria/*Case Studies {Education)/*Disadvantaged Youth/*Disadvantaged ' 
Environment/*Cultura11y Disadvantated/*Educati anally Disadvantaged/*Socia11y I 
Di3advantaged/*Early Childhood Education/*Te1evised Instruction/*Home Visits/*Mobi1ei 

Class^o^"^^/ =^ t 

Oregon Profile 7 describes a research project designed to determine the effective- 
ness of the Early Childhood Education Program in satisfying performance criteria | 
and specifications developed for the program- The project's intent is to develop a ^ 
cost-effectiveness system particularly appropriate for rural Appalachia communities.! 
The project established a television series to teach cognitive skills, a home 
visitor program to assist in its effective use, and a mobile classroom for distributlion 
of materials usually unavailable to children. Staffing patterns and backgrounds, ahd 
training and resources necessary are discussed as parmeters of the project* (AS) I 
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"Monitoring Innovation Processes in Education," in The Oregon Studies in Educational' 
RDD&E, Volume IV, Part One ' 
Green, Lee & Jones, Diane G. ' 
Monmouth, Oregon i 
Teaching Research * 

Proiect No. 0>Q701. Con tract No. Q EG-0-_70-4977 . 1972 |_ 

*Research Ski11s/*Educat1ona1 Researchers/*Research Projects/*Program Administration/ 
*Professional Education/*Questionnaires/*Samp1 ing/*Educational Innovation/*Teamwork/'^ 
*Data Processing/ I 

I 



Oregon Profile 8 depicts the MPIRE evaluation project. The MPIRE project aimedj 
to design an up-to-date, nationally applicable questionnaire to monitor innovation | 
in public schools. It also founded a Research Advisory Committee on Innovation , 
Processes in Education, which advised MPIRE and was to continue surveillance and i 
review. The staff of MPIRE included a director, two assistants, and a research ^ 
assistant. All consulted each other and outside colleagues frequently. Decisions ! 
were made by consensus whenever possible. Besides academic preparation, staff I 
considered understanding the project as a whole and being able to work with others I 
essential. The MPIRE staff recommended a combination of classroom instruction and f 
work experience as training for their kind of research. Training should include i 
work on problems with specific contexts and goals, requiring communication among team 
workers- (AS) ' 

I 
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'7ri-University Project on Behavioral Objectives in English, Grades 9-12,'* in The ^ 
Oregon Studies in Educational RDD&E, Volume IV, Part Two ' 
Smith, Clark A. ' 
Monmouth, Oregon ^ 
Teaching Research > 

Project No. 0-070K Contract No, OEG-0-70-4977, 1972 ^ ' 

^Research Skills/*Behavioral Objectives/*Research Projects/*English Education/*Mora1e/ 
♦Educational Researchers/*Professional Education/*Program Administration/*Measureme/jt 
Goals/ I 

< 

Oregon Profile No, 9 describes the Tri-University Project's goals. The i 
project aimed to formulate high school English teaching objectives in behavioral | 
terms. The project employed seven directors from three universities, many consul taiits, 
and clerical staff. During the project, the value of its aim was debated in a , 
professional journal. This challenge caused some defections among consultants, mor^ 
intense study by directors, and some changes of ©nphasis in the behavioral objectives 
formulated. The directors found awareness of others' feelings, abilities, and beliifs 
as important for success as verbal skills and knowledge of Enalish teaching. Four I 
inferences for training educational researchers are made: (1) Training should emph^isiz j 
not behaviorist technique itself but its principles. (2) Training should force | 
the student to focus on information sought rather than methodology, perhaps by 
making him play the client's role. (3) Researchers should identify the problem, thfen 
assume an "actively passive role." (4) If researchers understand project goals, thfy 
can acquire or borrow any needed expertise they lack. (AS) • 
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"Providing Wide Ranging Diversely Organized Pools of Instructional Objectives and 

Measures," in The Oregon Studies in Educational RDD&E, Vol. IV, Part Two 
Crowhurst, Norman H. ' 
Monmouth, Oregon I 
Teaching Research ' 

Prnjprt Nn, 0^0701, Contract No. 0EG--Q-70-4977 , 1972 | 

*Educational Researchers/*Professional Education/*Behavioral Objectives/*Measurement' 
Goals/*Research Skills/*Research Projects/*Program Administration/*Classification/ ' 
♦Evaluation/ J 



Oregon Profile No, 10 describes the Instructional Objectives Exchange project, i 
The project started out to collect and distribute instructional objectives used in | 
different grades and subjects. Because few could be collected, it now also aims , 
to generate new objectives, to exercise quality control, and to seek a better i 
classification scheme for objectives than subject/grade. The ongoing project is 
partly supported by growing sales of sets of objectives. The professional staff ! 
shares the work according to interest and urgency. Flexible use of objective sets, I 
especially the new attitudinal ones, such as, "self concept" and "tolerance," is I 
discussed. Classification of learning objectives is also discussed. Two possible i 
effects on the training of educational researchers are noted: 1, Papers on quality; 
control and classification prepared by the project to orient new members could become 
training packets for other professionals. 2. The objectives will someday be so ^ 
presented that teachers can generate their own test items, thus constituting a sort i 
of training program of wide impact. (AS) ^ 
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"Relevant Educational Application of Computer Technology," in The Oregon Studies 

in Educational RDD&E, Vol, IV, Part Two 
Green, Lee & Carl, Loring M, 
Monmouth, Oregon 
Teaching Research 

Pro ject N Q- Q-07Q1, Contract No. 0F rT n - 7n - 49 77> 1972 - 

♦Computer Oriented Programs/*Research Ski11s/*Educationa1 Researchers/*Professional 
Education/*Work Experience/*Research Projects/*Regional Laboratories/*Program 
Admi ni s t ra ti on/*Cu rr i cu 1 um Deve 1 opment/ 



Oregon Profile 11 describes the REACT project. The project aims were: (1) 
to create instructional packages to show high school students, teachers, and ad- 
ministrators how to use the computer to solve problems by calculation, simulation, 
or games; and (2) to start a clearinghouse of such instructional packages* The 
project works in two halves: the administrative training effort is developing a 
computerized miniature school system, while the teacher training effort is 
adapting packages for student use and reviewing curricula developed elsewhere for 
the clearinghouse. The director delegates responsibility and authority. Staff 
members freely share expertise. This project finds the doctorate of less real 
use than experience in education and with computers. It is suggested that training 
include simulated experience. (AS) 
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"The Integrated Career Development Curriculum," in The Oregon Studies In 

Educational RDD&E, Vol • IV, Part Two 
Myers, R. E. 
Monmouth, Oregon 
Teaching Research 
Project No, 0-07 01 , Contract No, 0EG>0-70-4977 , 1972 

♦Research Skins/*Educationa1 Researchers /*Program Administration/*Curriculum 
Deve1opment/*Research Projects/*Professional Education/*0ccupat1ona1 Guidance/ 
♦Vocational EducQt';on/*Curriculum Research/*Concept Teaching/ 



Oregon Profile 12 describes the developmental stage of the ICDC Project. TRe 
project's goals, staff, and working environment are described. The ICDC project 
aimed to provide a model curriculum, course content, and training materials, 
consisting of single-concept learning units, to help prepare rural highschoolers 
for careers. Students were to acquire not specific job skills, but rather attitudes^ 
and concepts useful in changing situations. The staff comprised a director in Utah,! 
a co-director tp.< Nevada^ and secretaries. The writing of single-concept units was i 
delegated to individuals supervised by the directors. A quality assurance panel of | 
consultants evaluated outputs at each step. There were to be preliminary and final , 
field tests in selected rural high schools. The work involved much co-ordination | 
and management. Staff considered their academic preparation useful, though none 
had received specific formal training for educational R&D. They recommended trainirig 
in evaluation techniques and curriculum development. They advised knowledge of PERlj 
and PPBS management techniques. Most important, they thought, were skills in 
communicating and dealing with people and a broad view of the world. (AS) 
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"The Development of Protocol Materials," in The Oregon Studies in Educational RDD&El 
Vol, IV, Part Two f 
Myers, R. E. ' 
Monmouth* Oregon i 
reaching Research ' 

Project No, 0-0701 . Contract No. O EG-0-70-4977 , 1972 ! 

♦Research Skills/*Educational Researchers7*Pr6gram Administration/*Professional ^ 
Educat1on/*Curriculum Development/*Research Projects/*Protocol Materia'!s/*Film ' 
Production/*Evaluation Criteria/*Behavioral Objectives/ 1 

t 

L 

Oregon Profile 13 describes the developmental stage of the PROTOCOL Project, i 
which aimed to film pupils showing "commitment to learning, constructive sense of | 
self, analysis, and evaluation (the last two from Bloom's Taxomony of Educational 
Objectives )" so that teachers could learn to identify these desired educational j 
outcomes in the classroom. The protocol idea came from Teachers for the Real World 
by B.O. Smith et al., 1969. The project also aimed to evaluate the feasibility \ 
of developing and using protocol materials. The staff consisted of a "Special f 
Resource Person," who proposed the project, a director and co-director, co- I 
ordinators at both schools where films were to be shot, and research assistants, i 
who wrote scenarios. Staff's schooling and experience had been mainly in the i 
field of education. A subcontractor was to produce the films. Still, staff needed 
more knowledge of production techniques than expected. On-the-job training became ' 
difficult. So the staff held a two-day retreat to clarify goals and roles. There \ 
were frequent staff meetings. The director attended training institutes for the . 
heads of similar projects nationwide. It is concluded that backgrounds in cinema ^ 
and curriculum development are indicated for workers on a project like PROTOCOL. 

(AS) : 
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"Development of a Multimedia Course in Leadership for the U.S. Naval Academy," ^ 

In The Oregon Studies in Educational RDD&E, Vol. IV, Part Two f 
Crowhurst, Norman H. ' 
Monmouth, Oregon , ' 

Teaching Research ' 

Project Nq 0-Q70K Contract No. OEG- 0-70 -4977. 19 72 : | 

♦Research Ski11s/*Educationa1 Researchers7*Program Administration/*Curricu1um 
Deve1opment/*Research Projects/*Research Design/*Mu1timedia Instruction/*Programmed' 
Instruction/*Curricu1um Research/*Research Problems/*Learnino Modal ities/*Evaluatioh 
Criteria/*Computer Assisted Instruction/ i 
\ 

Oregon Profile 14 is a case study of the Annapolis Project. The project aimed| 
to produce a multimedia course in "Leadership, Psychology and Management" for the j 
client, the Naval Academy, and to test the merits of different instructional media ^ 
and the importance of different instructional variables for the parent agency, I 
Westinghouse Learning Corporation. Media included print, pictures, film, audiotape, 
videotape, and the computer terminal. Some materials were syndactic, some programmed. 
The project's unusual design required everything the project produced to serve bothi 
research and deve1o|:nent purposes. Project and academy staff worked closely together 
developing course units and typing them out in the classroom, but the desire to conitrol 
all instructional variables for research conflicted with the effort to develop a | 
better curriculum. Each person working on the project had specific training or 
experience for his job. Most found experience in the Navy or at some service acadehy 
valuable. No general recommendations for training educational researchers are made|. 
(AS) i 
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"Study and Development of Automated Instructional-Materials-Handling Programs »" in ' 
The Oregon Studies in Educational RDD&E, Vol. IV, Part Two ' 
Hill, Herbert E. ^ 
Monmouth, Oregon i 
Teaching Research i 

Prelect No. 0-070K Contract No. OEG-0-70- 497 7 , 1972 ' 

*Research Skins/*Educational Researchers/*Program Admini strati on/*Research Projecti7 
*Professional Education/*Instructional Materials/*Library Technical Processes/ ' 
*Employment Qualifications/*Professional Education/*Computer Oriented Programs/ I 
* Information Systems/ i 

^ L 

Oregon Profile 15 depicts the developmental stage of the AIMS Project. The i 
AIMS project sought to design an automated system for acquiring and keeping track | 
of the Los Angeles Unified School District's instructional materials, from books 
to films to laboratory frogs. The system was to be adaptable to other California j 
districts and large cities elsewhere. The school district directed the project, bux 
subcontracted the work to Systems Development Corporation. SDC submitted each stepi 
to the school district for discussion and approval. Besides technical expertise, I 
library training, and writing skills, SDC staff needed the ability to communicate | 
verbally and to understand the client's requirements and constraints. Funds for i 
the project came through late. Most staff were assigned only part-time to the j 
AIMS project. The profile details resulting problems. It is suggested that 
educational research and development training include practice in the kind of team-' 
work wherein each individual contributes a finished part of the whole. Disserta- | 
tions might be written by teams. Apprenticeship and simulated work experiences, as 
well as formal instruction in management and writing, are recommended. (AS) 
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"Alternatives for Learning Through Educational Kesearcn ana lecnnoiogy," in Tne r 
Oregon Studies in Educational RDD&E, Vol* IV, Part Three i 
Hill, Herbert E,, Jones, Diane 6., and Carl, Loring M. \ 
Monmouth, Oregon i 
Teaching Research i 

Project No, 0-0701 , Contract No, 0EG-0-70> 4977, 197 2 i 

*Regional Laboratories/*Diffusion/*Educati onal Research/*ljualltications/^Management7 
*Information Dissemination/*Systems Development/*Formative Evaluation/*Writing i 
Skills/ I 



Oregon Profile 16 describes fFie ALbKi dittusion project, ine aleki project 
aimed to develop a system for diffusing information about successful educational 
developments in forms useful to school personnel. The development of a prototype 
system is summarized. The staff said that knowledge of curricula,- educational 
research, and statistics, and the ability to write clearly were important. At 
the time the profile was completed, ALERT was changing over to a task-oriented 
team structure, which gave junior staff unwanted managerial responsibilities and 
made interpersonal skills more important. It is concluded that training for 
research and development should include writing, teamwork, and management techniqi 
(AS) 
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"Paul L. Dunbar Community Learning Center, in The Oregon Studies In Educational 

RDD&E, Vol, IV, Part Three 
Crowhurst, Norman H. 
Monmouth, Oregon 
Teaching Research 

Project No, 0-Q701 , Contract No, OEG-0-70 -4977 , 1972 

*Diffusion/*Disadvantaged Youth/*Educational Research/*Educat1ona1 Researchers/ 
*Management/*Educational Innovation/*Teacher Education/*CommunTty Involvement/ 
*Curriculum Study Centers/*Language Arts/*Music Education/*Social Studies/ 



Oregon Profile 17 depicts the Dunbar Center diffusion project. The project's 
goals, philosophy, staff, and activities are described. The Dunbar Center adapted, 
tried out, and locally promulgated innovative curricula, such as, Taba social 
studies, AAAS science, IPI math, Suzuki music, and others. In support of classroom 
programs, the Center devised teacher training, guidance, and parent-community 
involvement programs. The profile's appendix reproduces the raw data— statements 
from interviews--which form checklists for these activities. The Dunbar Center, 
in a disadvantaged neighborhood, gave the feeling of successful learning. The 
parent-community involvement program was an important reason. Teachers and 
curriculum adapters regularly exchange roles. Such experience, it is suggested, 
might improve teacher and researcher training, (AS) 
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"Educational Resources Information Center Processing and Reference Facility," in 

The Oregon Studies in Educational RDD&E, Vol. IV, Part Three 
Crowhurst, Norman H. 
Monmouth, Oregon 
Teaching Research 

Project No. 0-070K Contract No. 0EG-0-70-4977. 1972 



*Iriformat1on Science/*Information Centers/*Clearinghouses/*InforTriation Dissemination^ 
*Management/*Abstracting/*Indexes/*Computer Oriented Programs/^Educational Researchefs/ 
*Technical Writing/*Professional Education/ 



Oregon Profile 18 depicts the ERIC Processing and Reference Facility, a diffusion 
project that, with computer assistance, published several periodical guides, listing^ 
and indexes to educational journals, reports, etc. It also published its descriptor 
thesaurus, acted as a clearinghouse, and answered queries. The project's rationale 
and some problems of information systems design are explained, A list of twenty 
ERIC clearinghouses and a chart of the ERIC organization are presented. The 
project staff is described and charted. Its schedule is diagrammed. The 
project emphasized speed, accuracy, and unit cost effectiveness. Appreciation 
of the work's importance kept morale high. Technical computer personnel found 
specific coursework their best training. Abstractors, indexers, and managers found 
an understanding of language usage most important. (AS) 
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A Case Profile of a Project Titled: Children's Television 

Workshop (CTW Project) 
Children's Television Workshop Corporation 
Monmouth, Oregon 
Teaching Research 

Project No. 0-070K Contract No. OEG-0-70-4977, 1972 

^Personnel Managenient/*Educationa1 Television/*Preschool Education/*Fi 1m Production/ 
*Scripts/*Evaluatiori Methods /*Public Re1ations/*Curricu1um Development/*Diffusion/ 
*Educatioria1 Research/*Management/*Financia1 Support/*Community Relations/ 



Oregon Profile 19 is a case study of the Children's Television Workshop. CTW | 
produced the Sesame Street educational program for disadvantaged preschoolers and . 
was preparing a remedial reading program for gradeschoolers. It supervised the design 
and marketing of educational toys and materials. Its publicity and community involve- 
ment programs included the publication and distribution of a magazine. CTW's work ' 
exemplifies diffusion based on continuous in-house research, development, and ! 
evaluation. CTW's working environment is described. Its schedule is displayed. I 
205 outputs necessary for its work are listed in an appendix. Of these, thirteen i 
products and forty-six management outcomes are described and classified, and their , 
interdependencies are sketched. A roster and organization chart of CTW's staff are , 
presented. Few held advanced degrees, but their professionalism and understanding 
of and commitment to CTW's goals are judged high. Nine reasons why the organization 
works well are listed, with careful selection and on-the-job training of personnel | 
being foremost. On-the-job training programs in TV production and writing are 
outlined. Their truly goal-directed context is judged most important. (AS) 
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"The Assessment of Exemplary Reading Programs," in The Oregon Studies in Educational 

RDD&E, Vol, IV, Part Three 
Myers, R. E, 
Monmouth, Oregon 
Teaching Research 

♦Technical Writing/*Interviews/*Evaluation/*Personnel Management/*Dif fusion/ 
♦Reading Instruction/*Educational Research/ 



Oregon Profile 20 describes the American Institutes for Research (AIR) Assessment 
of Exemplary Reading Programs^ classed as a diffusion project. The AIR project's 
goals, staff, and working environment are described. The subtasks, skills, sensi- 
tivities, and standards for each of seven production and eight management outputs 
are listed as extracts from profile interviews. The extracts form checklists on 
report writing, report production, interviewing, evaluation, personnel management, 
and relations with financial sponsors. The AIR project aimed to publish evaluative 
reports and ranked lists of innovative programs in reading instruction and childhood 
learning. The reports were based on site visitation, so skill in interviewing was 
important. Skill in jargon-free writing was even more important. It is recommended 
that specific writing abilities be evaluated before a person is hired for educationa 
research, (AS) 
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The Development of a Pilot Library of Cassette Tapes Dealing with Recent 

Advances in the Strategies and Features of Educational Research 
Popham, W. James 

1126 16th Street, N. W,, Washington, D. C. 20036 
American Educational Research Association 

Hf^cpmher, IQ?!, OFR^n^70-47qf^ fBPO^, 48 p. ^ 

^Educational Researchers/*Indivi dual Instruction/*Instructional Materials/^Instruc- 
tional Media/*Magnetic Tape Cassett^^s/*Professional Continuing hducation/*Research 
Skills/*Summdti ve Evaluation/*Tape Record1ngs/*Training Techniques/ 



This project was designed to produce a series of half hour cassette autiotapes 
to be used as instructional materials to improve the skills of educational 
researchers. In addition to developing the cassette tapes, the project was to 
assess, at least initially, the degree to which such tapes could serve as an 
effective vehicle for the continuing education of such research personnel. 
To accomplish this the cassette tapes developed during the project were turned 
over to the American Educational Research Association (AERA) in order that they 
could be distributed commercially by AERA, thereby testing the economic 
feasibility of developing and distributing such tapes. 

In all, 14 separate tapes were prepared by some of the nation's leading 
educational researchers, thereby totally more than seven hours of instruction. 
The first 10 of these were distributed by AERA during 1971, with more than 
1,400 tapes being sold in less than eight months. It is apparent from these 
sales figures that educational researchers will, indeed, purchase such tapes 
in sufficient quantities to make them economically feasible to produce. 

An external evaluation of the purchaser's satisfaction with the first ten 
tapes was conducted independently by Professor Jerry L, Brown of Indiana 
University. In general. Professor Brown concluded that the tapes were judged 
useful by their purchasers and that most purchasers reported a willingness 
to acquire more tapes. A number of procedural questions regarding usage of 
the tapes were explored as a consequence of responses to a questionnaire 
distributed to tape purchasers. 
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Research for Curriculum Improvement 

Association for Supervision and Curriculum Development, NEA 

Association for Supervision and Curriculum Development, NEA 

-1957 ' ^ 

*Annotated Bibliographies, Cooperative Planning, Curriculum Development/ 
♦Experimental Curriculum/^Curriculum Research/*Classroom Research, Research 
Design/*Educational Research/*Research Methodology, Research Projects/ 
♦Research Skills, Institutional Research/ 



This volume, which is the 1957 yearbook of the Association tor supervision 
and Curriculum Development of the NEA, provides a history of curriculum 
research, describes the research process, discusses the researcher himself, 
and details various aspects of cooperative curriculum research in the school 
setting. The work is intended to fulfill the needs of all educators con- 
sidering, or actually, doing research in instructional improvement. 
Initially provided are a discussion of the theoretical principles which must 
be considered by curriculum researchers and a history of curriculum research* 
Section II covers the phases of the research process: 1) problem identifi- 
cation; 2) determining work hypotheses, 3) selecting and handling data, and 
4} obtaining meaningful results from the data. Section III asks the reader 
to consider himself as not only a consumer, but also as a producer, of 
research; and discusses the necessity of good human relations when conducting 
research. The final section discusses the following aspects of conducting 
curriculum research in the school: organization and management of the 
research process; inhibitors in conducting research; and ethical considerations 
when using students in educational experiments. Appendixed are an annotated 
bibliography on curriculum research and a report on the first Cooperative 
Curriculum Research Institute. (MG) 
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Instructional Product Research '. " 

Baker, Robert L, Schutz, Richard E., ed. 

11300 La Cienega Boulevard, Inglewood, California 90304 

Southwest Regional Laboratory for Educational Research and Development 

1972, 314 p, 

*Data Analysis, Research Proposals, Instructional Materials, Program Design/ 
^Research Design, Educational Experiments, Research Projects/*Educational 
Research/*Research Methodology, Research Skills, Research Tools/*Evaluation 
Techniques/ 



The material in this book presents various research tools and techniques 
relevant to the research process and useful for developing instructional 
products. It is intended for the user of research and for novice in 
educational research and development. Initially discussed are the classi- 
fication of variables, the use of status, association and experimental studies, 
and interpreting the results of educational research studies. Identifying 
and selecting dependent and independent variables, and basic problems to 
avoid in their selection are discussed. A brief discussion of the research 
problem, guidelines, and sources for reviewing relevant literature are 
provided. In preparing a research proposal, the reader learns to outline 
the requirements and conditions of its basic components: the title, the 
research problem, procedures, and logistics. The interrupted time series 
design and the following control group designs are discussed as being 
very suitable for evaluative investigations in schools: non-equivalent, 
pretest-posttest, and posttest only. A strategy is provided for choosing 
appropriate statistical procedures for analyzing research data and the 
reader is familiarized with computer terminology and the use of library 
computer programs for statistical analysis. Formulating the requirements 
and conditions of the following research report components are discussed: 
title, research problem statement, method, results, discussion, conclusion, 
summary and abstract. (MG) 



Southwest Regional Laboratory for Educational Research and Development ' 
11300 La Cienega Boulevard * 
Inglewood, California .90304 | 

L 



ERIC 



FWL 022 0169 

A Training Program in Educational Development, Dissemination, and tvaluatlon 

Banathy, Bel a 

1855 Folsom Street, San Francisco, Califor,nia 94103 

The Far West Consortium, Far West Laboratory for Educational Research & Development 



*Coran)uni cation Skills/*Curriculum Development/*Data Coilect1on/*D1ffusion/*Education^l 
Development/*Evaluation/*Field Experience/*Formative Evaluation/*Indi vidual Instruction/ 
^Information Retrieval/*Instructional Materials/*Marketing/*Professional Continuing 
Education/*Professional Training/*Program Description/*Program Planning/*Technical R^- 

^^^^H^ls comprehensive, transportable program is designed to train personnel in | 
educational development, dissemination, and evaluation (DD&E) skills. The 
content of the program covers nine competence areas: planning and design, 

information/data collection and organization, communication skills, developmental I 

engineering, evaluation, analysis and definition, dissemination and marketing, ' 

management, and dissemination of educational information. Training materials for I 

each area are labelled a "series." Each series consists of from four to seven I 

"modules" or self-instructional units focusing on sets of related skills. I 

Competence assessment batteries, consisting of instruments such as trainee , 

and supervisor ratings, job knowledge tests, simulation exercises, and product , 
rating scales, are provided for each of the competence areas. The program 

features an instructional pattern called a triad, which consists of the student, i 

an Instructional Resource Manager (IRM), and a work supervisor. The student I 

works with the instructional modules, performs activities and exercises, and, ^ 
at the same time, applies skills in an actual work situation. The IRM functions 

as a guide and resource person, assigning modules and reviewing student progress ' 

on a work plan and on achievement of assigned modules. The work supervisor ^ 
oversees the student's job activities, provides on-the-job instruction as 

needed, and provides feedback cind evaluation to the IRM. Target audiences for ' 

the training are personnel planning to enter or already in DD&E positions. There ' 

are three suggested levels for using the training: (T) a comprehensive, degree- ^ 
oriented program coordinated at a college or university; (2) use of series or 

clusters of modules as required by the learner; or (3) use of individual modules , ' 

as required. About 30 printed modules, competence assessment instruments for each < 

module, catalogs of instructional resources and assessment materials, and guides \ 

for quality control and implementation of program will be available in field i 

test form by Fall 1973. ' 
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The Educational Information Consultant: Skills in Disseminating Educational 

Information 
Banathy, Bel a 

1855 Folsom Street, San Francisco, Califor/iia 94103 

Far West Laboratory for Educational Research and Development 



♦Change Agents/*Communicdtion Problems/*Information Dissemination/*Infonnation Meeds/ 
^Information Retrieval/*Information Seeking/*Instructional Materials/*Problem Solving 
*Role Playing/*Simulation/*Training Techniques/ 



The EIC package provides skill development training for a role in education of 
a "middleman" who knows how to find and make accessible educational information 
where it is needed. The program is based on a model which delineates five major 
processes for this role. Participants learn to: analyze and define an educator 
client's information problem; plan and execute an information search; screen, 
sort, and package the information; communicate it to the client; and evaluate how 
effective the service has been. The training is skills-oriented and learner- 
centered. The exercises feature group and individual activities, including role 
playing, problem solving, simulation, and. decision making. Student self- 
evaluation exercises and skill evaluation criteria and devices are included in 
the package. The training is available in three forms: Course, Institute, 
and Learning Team. Its target audience includes educational information 
dissemination personnel at school, district, county, regional, or state levels, 
e.g., information specialists, librarians, curriculum or subject matter specialists, 
instructional materials center personnel, graduate students. The training can be 
used for preservice or inservice training as a 45-hour graduate course, as a 10- 
day workshop, or as an independent study program for a team of three. Materials 
include a training manual (one per student) and an Instructor's package, with 
Instructor's Guide, model information packet, communication game, two filmstrips, 
and two cassette audiotapes. Access to an ERIC collection and to a library or 
information center is required. The Course and Institute can be directed by any 
university instructor; a background in information utilization is helpful, but not 
required. The Learning Team is self-administered. Course and Institute will be 
available in Spring 1973; a field test version of Learning Team will be available 
also. 
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Appendices, Sample Material for the FEHR-Practicum Game, Final Report 

a Computerized Game to Simulate Experience in Ed. Research & Evaluation 
Collet, LeVerne S. 
Ann Arbor, Michigan 
The University of Michigan 

August. 1971, OEC-0-70-4773 (5 20) 

♦Computer Assisted Instruction/*Educational Games, On-Line Systems, Simulation/ 
♦Training Techniques/*Fonnative Evaluation, Decision Making, Evaluative Thinking, 
Evaluative Techniques, Research Skills/*Educational Research/ 



This project report details the major components, characteristics and use 
of the FEHR- (formative evaluation and heuristic research) PRACTICUM game. 
This computerized game is intended to provide a wide range of practical 
experience in educational research and evaluation without the expenses and 
time commitment involved in actual research. The game is designed for use in 
educational research training programs and for in-service training for researchers, 
curriculum developers, etc. The players are given a problem of a hypothetical, 
operational school system and must plan, design and execute a series of field 
studies. Experience is to be gained in gathering and analyzing data to make a 
practical educational decision. Feedback is provided on the adequacy of the 
adequacy of the players' decisions. Practical experiences to be derived from 
playing the game are discussed. The four major program components, information 
bank, data generator, message generator, and in-service training unit, are 
described and their technical details enumerated. The following are appendixed: 
1) player's introductory booklet; 2) statistics on Fair City, the hypothetical 
environment; 3) player's orientation booklet stating the problems and rules; 
4) game manager's script to accompany the player's orientation booklet; 5) current 
contents of the information bank; 6) current status of the FEHR-PRACTICUM 
problems; 7) list of common variables; and 8) player's problem booklets for 
eight game problems. (MG) 
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A Programmed Introduction to Statistics, 2nd edition 

Elzey, Freeman F. 
Belmont, California 94002 
Wadsworth Publishing Company 

iq71, -^»R p, 

*Statistics/*Statistical Analysis/*Programmed Instruction/ 



This book is designed to introduce one to statistical techniques and their 
application through programmed sets of instruction. It is divided into twenty- 
five sets, each containing (a) an introduction, (b) the programmed instruction 
portion, and (c) a series of exercises. The introduction of each set states 
the objectives of that chapter; the programs are divided into frames, each 
requiring an answer to a fill-in-the-blank statement; the exercises provide 
an opportunity to test what the sets have presented. Formulas, tables, and 
a glossary of statistical symbols are provided. Topics covered include: 
organization of data, measures of central tendency (mode, median, mean), 
comparison of measures of central tencency, percentiles, graphic renresentation 
of frequency distribution, measures or' variability (range, semi-interquartile 
range, average deviation, variance, standard deviation), relationship between 
population and sample, introduction to the normal distribution, probability 
and the normal distribution, sampling error, estimation of population variance, 
estimation of the standard error of the mean, 95% confidence interval, 99% 
confidence interval, t^ distribution, null hypothesis, and standard error of 
the difference between means, t ratio for independent and non-independent 
means, one-and two-tailed tests, type I and type II errors, analysis of 
variances, f test for two variance estimates, scatter diagrams, introduction 
to correlation, Pearson product-moment correlation, regression. Spearman's 
rank order correlation, chi square (single sample and multiple samples). 
(MB) 
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Developing Programmed Instructional Materials, A Handbook tor Hrogram f 

Writers \ 

Espich, James E., Williams, Bill » 

2165 Park Boulevard, Palo Alto, California 94306 i 

Fearon Publishers, Inc. i 

1967, 137 p. ^ I 

Programmi ng/*Programnied Instruction, Programmed Ma'terials, Braliching, Constructed ^ 

Response/*PrograTnniers/*Prograniming Methods, Feedback, Auto-Instructional Methods, i 

Sequential Programs, Educational Psychology, Evaluation Methods, Evaluation I 

Techniques, Fixed Sequence/ i 

This volume briefly discusses the theories behind programmed Instruction, , 
describes the preparatory steps required for programming, details the major 

programming techniques for writing various types of test frames, and suggests ' 

procedures for editing, testing, and test analysis. Although mainly an ■ 

introductory text for program writers, it is also oriented towards individuals I 

involved in evaluating programmed instructional materials. Theories of behavioral | 

psychology are discussed as the basis for programming techniques. Three basic i 

questions are then detailed which should be posed to determine the feasibility | 

of writing a program. The programmer then must: establish objectives; , 

examine the scope, level and results of material currently utilized; discuss j 
the program with a subject expert; and diagram the material to be presented 

using flow charts or schematics. The following program construction techniques I 

are discussed: discrimination frame sequence; constructed response frame j 

sequence; branching frame sequence; retrospective chaining; BABOON frames; . 
adjunct programming; and adjustive techniques such as branching programs, gate 
frames, remedial loops, and secondary tracts. Editing for programming techniques, ' 

technical accuracy and composition are discussed. One-to-one, small group, > 

and field testing are discussed as experimental testing techniques. After * 

testing, the minimum effort required to comprehensively revise a program is ' 
detailed. (MG) 
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A Guide to Educational Resources 

Far West Laboratory for Educational Research and Development 
1 Garden Circle, Hotel Claremont, Berkeley, California 94705 
Far West Laboratory for Educational Research and Development 

^Annotated Bibl iographies/*Bibl iographies , Information Sources, Information 
Services, Reference Books, Information Centers/*Literature Guides, Information 
Systems, Bibliographic Citations/ 



Intended primarily for the educational Information consultant, this brief 
annotated bibliography is designed to selectively cover standard sources of 

information for preliminary literature searches and those literature tools ' 

and services available for current awareness of activities, products, sources ' 

and innovations. The full bibliographical citation and the work's price are ' 

detailed, when appropriate, for each Item covered. The work describes i 

reference tools available for: general fact finding; social science sources; \ 

guides to educational literature and research; basic reference in education; | 

statistics; directories; guides to current literature, Office of Education , 
projects, funds, and selection of periodicals; and guides for products and 
audiovisual materials. Information is provided on the Educational Resources 

Information Center (ERIC), Its twenty clearinghouses, their mailing address, I 

and areas of specialization. Also listed are the mailing addresses of regional j 
laboratories, university-sponsored research and development centers, and several 

related agencies which are funded by the U. S. Office of Education, Several * 

non-conventional information services for current awareness and retrospective ' 

literature searches are also annotated. (MG) » 
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Program Evaluators Handbook, Measurement 

Farr, David, Subkoviak, Michael, J- 

Albany, New York 12224 

The State Education Department 

1969, 30 p, 

*Standardi2ed Tests/* Test Construction/*Test Selection, Observation, Item 
Sampling/*Item Analysis/*Measurement Techniques, Institutes {Training Programs), 
Evaluation Methods/*Evaluation Techniques/ 



This manual is the result of a review and training session tor strengthening \ 

evaluative techniques. Conducted by the State of New York, the session was j 

specifically concerned with Title III of the Elementary and Secondary Education ' 

Act of 1965, although the material is generalizable. The material is presented ! 

such that the original training session is replicable with time devoted to ' 

lecture, class discussion, and practice exercises. The manual has five units ! 

with expository text, an outline for use in lecture and/or class discussion, I 

and exercises listed for each unit. The five unit topics are: 1) measurement- | 

purposes, ideals, possibilities; 2) defining measurement domains, 3) person , 

and item sampling; 4) test of item selection; and 5) objective observation- l 
A Selected List of Standardized Tests details each test's name, publisher, 

grade levels covered, testing time, major subject areas measured, and where ' 

it is reviewed in the Mental Measurements Yearbook. General and specific 1 
suggestions for test item writing are given for short-answer, true-false, 

multiple choice, and matching forms- (MG) ' 
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Assessment and Evaluation Handbook, Title 1 ^ 

Finkelstein, Elsie, L. ' 
Albany, New York 12224 i 
The State Education Department, Office of Research and Evaluation » 

; 

^Annotated Bibl iography/*Evaluation Techniques/*Program Evaluation, Performance 
Tests, Performance Criteria, Evaluation Criteria/*Evaluation Methods/*Summative ' 
Evaluation/*Formative Evaluation I 

I 



This manual basically presents an overview of the evaluation process and 
extensively covers numerous evaluation planning charts and assessment devices 
for the evaluation of specific programs. It was developed by the New York 
State Education Department and is intended as a guide to assist local school 
personnel in assessing the effectiveness of programs and projects initiated 
under Title I of the Elementary and Secondary Education Act of 1965. Initially 
discussed are the purpose of evaluation, the planning and development of 
evaluation procedures, and six basic evaluation designs. Utilization of 
the State Pupil Evaluation Program and state agencies available for assisting 
project personnel are briefly described. Twenty-nine samples of evaluation 
planning charts are then included for use as guides or suggestions when 
planning evaluation procedures. Each chart has four headings for listing 
the general objective, specific objectives, pertinent behavioral criteria, 
and evaluation procedures. The sample charts are on the following topics: 
educational communications; general education; handicapped children; health, 
physical education, and recreation; humanities and the arts; industrial arts 
education; occupational education; pupil personnel services; school supervision; 
and teacher education. Several of the sample charts include examples of various 
relevant evaluation devices such as questiornaires , checklists, etc. An 
annotated bibliography is included which lists numerous source materials 
pertaining to school program evaluation. (MG) 
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Planning a Test--A Filmstrip Unit on Basic Measurement Principles 

Fremer, John J. , Jr. 
Princeton, New Jersey 08540 
Educational Testing Service 



^Criterion Referenced Tests/*Educational Testing/*Evaluation/*Filmstrips/*Instructionfil 
Material s/*Measurement/*Measurement Instruments/*Questionnaires/*Research Skills/ i 
*Test Construction/nest Interpretation/*Test Reliability/*Test Selection/*Training 
Techniques/ ' 



A filmstrip with associated audio track has been developed to cover the major 
planning steps in the development of a measurement instrument such as a test or 
questionnaire. The filmstrip addresses the following six questions: why am I 
testing, what should I test, whom am I testing, what kinds of questions should I 
use, how long should my test be, and how difficult should my test be? A set of 
supplementary print materials cover the following issues: learning how to 
develop tests, obtaining information about tests, preparing a test plan, kinds of 
test questions--advantages and disadvantages, reliability, and criterion-referenced 
tests. The supplementary materials provide an expanded treatment of some of the 
issues raised in the filmstrip, but their primary function is as a guide to 
appropriate literature for those individuals seeking an intensive treatment of 
topics related to their particular interest and needs. No student testing or 
evaluation materials are included in the package. No previous training in 
measurement is assumed. The projected target audience includes persons engaged 
in or receiving training in evaluation, such as students in introductory 
educational psychology, measurement, or research training, or teachers, 
administrators, or other educational personnel. The unit can be used in graduate 
or undergraduate level courses or for inservice training courses or workshops. 
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Doing a Literature Search in an Information Center 

Garman, Marcia B. and Kathleen Mitchell 

590 Hamilton Street, Redwood City, California 94063 

Educational Resources Center 

Q-^P^ ^ 

*Search Strategies/^Information Retrieval/*Information Seeking, Information 
Needs/*Inforniation Sources, On-Line Systems, Bibliographies, Thesauri, Subject 
Index Terms, Reference Materials/ 



This course is intended to teach individuals who want to become educational 
information consultants different methods of doing a literature search in an 
information center. Through numerous practice exercises, activities, readings, 
tapes, slides, and films, the reader is introduced to both traditional and non- 
traditional literature sources. The course's objectives are: 1) to identify 
various relevant glossary terms; 2) to transform a given search request into 
applicable descriptor terms; 3) to use manual techniques to select ten relevant 
documents, for a given search request, from ERIC publications and "fugitive" 
information catalogues; and 4) to use computer techniques to select ten relevant 
documents, for a given search request, from DIALOG, an on-line retrieval system 
for educational materials. Numerous self-tests are included, with their answer 
keys for the instructor. Equipment, materials, and facilities needed are 
listed. (MG) 
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Considerations in Developing Simulation Materials for the Training of T 

Educational Research and Research-Related Personnel i 

Goodwin, William, L., Worthen, Blaine, R. « 

Washington, D.C. i 

American Educational Research Associat-Ion i 

July. 1970 . 0EG-0>9-180240-3757 (010), 42 j^^ • 

Educational Research-Educational Researchers/*Simulation/*Training/*Teaching ^ 

Techn1ques/*Educational Programs/*Research Design/*Personnel Needs/*Educational ' 

Development/ ! 

i 



This report explores the use of simulation technique in the training of 
research and research-related personnel. It attempts to define topical 
areas and forms of most practicality for simulation in this field. The 
paper is divided into four parts: 1) the use of simulation for instruc- [ 
tional purposes, and its use for training personnel in educational roles 
in particular; 2) suggested steps in designing a simulation; 3) areas in 
RDD&E most favorable for a simulation format; and 4) issues pertinent to 
the role of simulation in the area, of research training. Simulation techniques | 
for instructional purposes are reported in vocational and medical education , 
(teaching and assessment), but thus far use of simulation for training in the , 
functional areas of RDD&E has generally been overlooked. Eight steps for 
designing simulations are suggested. Tables for the potential utility of I 
simulation in research, research-based development* diffusion, context I 
evaluation/situations analysis, program planning/input analysis, process : 
evaluation/program monitoring, and outcome evaluation training are given 
with some indication of each areas 's susceptibility to simulation, probable ' 
amount of work involved, and present availability of simulation materials. ' 
The basic issue of this paper is the extent to which simulation is an appro- ' 
priate instructional mode for training in RDD&E. Other issues, in the areas | 
of development and administration, include the lack of basic research that 
bears on the development of simulation, the relative effectiveness and j 
efficiency of simulation compared to other instructional alternatives, in- 
formation on how the simulation development is to proceed, resource costs of ' 
simulations, possible uses of simulation, whether or not the simulation should ' 
be designed to replicate a portion of or a total system, incorporation of > 
simulatlon(s) into research training programs, how long the simulation should t 
be, whether practice is spaced or mci*>sed, the size of the group involved in i 
the simulation, and the consideration of the merit of a simulation carefully i 
designed and validated to be used in a controlled setting. The authors feel » 
the merit of simulation in the area of training for RDD&E personnel is worth ' 
further study. (MB) , 
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So You Have to Write a Technical Report; Elements of Technical Report Writing 
Gray, 



♦Technical Reports, Reports/*Technical Writing, Writing Ski lls/*Abstracting, 
Descriptive Writing/*Expository Writing/ 



This volume, written for anyone who has to write a technical report, is 
primarily concerned with those aspects and principles which are applicable 
to the preparation of all technical reports. The work's objectives are 
to provide basic guidance in identifying the audience for the report, 
organizing material and planning the document, and writing the full report. 
After defining a technical report, it's principal function, chracteristics, 
and various types of technical reports are detailed. Technical report 
writing techniques are identified and discussed and their advantages and 
limitations are pointed out. Also, possible good alternatives are suggested 
for specific cases where they would be preferable. The initial steps of 
report writing discussed are analyzing the overall problem, assembling necessary 
data, preparing a working outline, and thinking about the report presentation. 
The parts of a report are described in the following order in which they should 
be written: 1) introduction; 2) main body; 3) conclusions and recommendations; 
4) appendixes, references, footnotes, and table of contents; 5) abstract; 
and 6) title page. For each report part, its function, material to be included, 
location in the report, possible formats, style, and organization are discussed. 
The following characteristics of good technical report writing are then 
considered: completeness; conciseness; veracity; restraint; clarity; and 
general appearance. Finally, a summary of the volume is detailed in outline 
form. (MG) 
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A Technology for Developing Instructional Materials 

Gropper, George L. 

710 Chatham Center, Pittsburgh, Pennsyl ve^nia 15219 
American Institutes for Research 



*Behavioral Objecti ves/*Formative Evaluation/*Indi vidua! Instruction/^Instructional 
Design/*Instructional Materials/*Instructional Programs/*Material Development/ . 
*Simulation/*Task Analysis/*Test Construction/*Training Objecti ves/*Training Techniqiies/ 



This self-instructional program provides training tor developers of instructional, 
materials. Trainees engage in active practice of all the major tasks in the 
instructional development process, including: task analysis, stating objectives, 
developing tests, planning simulations, formulating strategies, developing materials,! 
trying them out, and revising them, A Handbook is available to serve as an aid ' 
during learning, as a job aid immediately after program completion, and as a 
reference source as the developer gains experience. Active practice of small 
development steps is provided in an accompanying workbook. Active practice of 
large tasks (combining the small steps) is provided in a final exercise volume* 
Final student evaluation exercises are included in the materials. The program is 
targeted toward beginning and intermediate audiences, with some portions for advanced' 
audiences. It requires about 30-50 hours of self-paced instruction and can be used 
in academic, industrial, or military settings. The printed materials include a ( 
Handbook, with 11 subvolumes. Workbook, Final Exercise, User's Manual, and Orientation. 
No equipment is required. These are scheduled for publication within a year. 
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Training Materials for Research, Development and Diffusion Training Programs 

Cuba, Egon 6., and Gephart, William J. 

U. S. Department of Health, Education and Welfare, Office of Education 

1970. 767p. 

*Professional Training/*Material Development, Measurement Techniques, Research 
Design, Research Methodology, Diffusion, Developmental Tasks, Evaluation 
Methods, Evaluation Techniques, Statistical Analysis, Questionnaires/ 



This volume is the result of a project which identified and collected t 

semi -developed materials and processes that would be useful for educational . 

training programs in research, development, diffusion, and evaluation. ^ 

Materials were identified through: 1) reviewing the literature; 2) indirect I 

contact via newsletters, announcements at meetings, etc.; and 3) personal ' 

contact with individuals involved in research instruction. The 327 partially I 

developed materials concern a wide range of research topics; but the majority I 

were on statistics, measurement, and research process in general, and research I 

design. The volume contains the following information on each of the materials: i 
author; address; title; topical focus; purpose; physical description; restrictions j 
to use; and general character. Many of the documents are dependent upon other 

printed material or oral presentations; conceptual criteria is lacking for ^ 

judging quality; and student entry, or terminal, behaviors and evaluative I 

data are lacking. In addition, the project was concerned with the possible ^ 

refinement, development, and adaption for use of these semi -developed ^ 
materials. Four instructional materials were refined as exemplars: 1) a 

coordinated set of transparencies and text on the concepts and activities ♦ 

in evaluation; 2) a manual and practice materials for identifying and des- | 

cribing segments of the school day; 3) a description of the elements in , 

questionnaire construction and an illustratory case history; and 4) a set ^ 

of flow charts and instructional materials for assessing the methodological ^ 

adeqjacy of completed research. These four developed materials are contained ^ 
in their entirety in the appendix. (MG) 
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An Interviewing Training Module 

Haller, Emil J. 

Ithaca, New York 14850 

Department of Education, Cornell University 



*Data Conection/*Field Interviews/*Films/*Instructional Material s/*Interviews, 
Questionnaires, Question Answer Interviews/*Research Methodoloqy/*Research Skills 
Role Playing, Surveys/*Training Techniques/ 



This instructional package is designed for general use in training individuals ort 
groups in the basic techniques of conducting survey research interviews. It can also| 
be used to acquaint students with the interview as a data collection device--e.g. , in, 
a research methods course--without unnecessarily concentrating on the practice of | 
specific skills. The module package consists of the following components: three 
stop-action films; a set of structured role-playing activities to be used during the 
breaks in the films; two copies of professionally developed interview schedules which" 
students administer outside of the training sessions; a set of question-by-question 
specifications for the schedules to assist in their usage; and a student manual which i 
explains the basic skills involved in conducting an interview. There is a procedural 
guide for an instructor. Each component is designed to supplement the others, providing 
students with the skills they need, an opportunity to practice these skills in a 
relatively controlled setting, and an opportunity to practice them "in the field." I 
Student evaluation materials are supplied with an instructor's manual. The entire 
package is nearly self-instructional, requiring little preparation on the part of the, 
instructor. Although the module is designed to be used in three, one-hour training 
sessions, plus out-of-class assignments, this format may be altered at the instructor) 
option. The target audience includes persons in a college or university, educational 
or public health agency, or private corporation who need training in interviewing 
techniques. 
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A Guide to Innovation in Education ^ 

Havelock, Ronald, G. ' 
Ann Arbor, Michigan i 
The University of Michigan, Institute for Social Research * 

1970, 166 p, ^ !_ 

^Educational Innovation/*Educationa1 Change/*Change Agents/*Case Studies (educatioH) 
/educational resources, educational problems, educational planning., educational ' 
strategies, manuals, resource guides/ 1 

I 

. : . L_ 

This manual contains information about how change occurs and how educators i 

who work for change (change agents) can organize their work to make successful . 
innovation take place. The orientation is toward problem solving by and for 

the user through* ef^feetive use of resources. Material included is based on an I 

analysis of 1,000 studies of innovation and knowledge utilization in education ^ 

and other fields. The process of change and innovation is divided into six ' 

stages: 1) building a relationship (between change agent and client); 2) I 
diagnosing the problem; 3) acquiring relevant resources; 4) choosing the solution; | 

5) gaining acceptance; and 6) stabilizing the Innovation and generating self- ^ 

renewal. The first part of the Guide gives case studies of projects in school i 
settings to illustrate the process. Part two, the main text of the manual 

illustrates the six stages of planned change with quotes from prominent authors ^ 

and references to the case examples. Supplementary resource information is ( 

covered in the third part of the manual and includes sections: A. strategies . 
and tactics: a glossary and guide to selection, which goes over common change 

strategies and tactics; B. major information sources in education which gives ^ 

bibliographic and annotative services, directories and indices; consulting ' 

organizations; and government agencies; and C. a guide to the literature on | 

planning of change in education with annotations and author and subject ^ 
indices. The guide is arranged for easy reference. (GS) 
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Training for Change Agents; A Guide to the Design of Training Programs ^ 

in Education and Other Fields i 

Havelock, Ronald G. » 

Ann Arbor, Michigan i 

The University of Michigan i 

197K 206 p . ' 

Educational Programs/*Training, Laboratory Traini'ng, Teaching Mode1s/*Training ^ 

Objectives, Trainers/*Change Agents/*Institutes (Training Programs)/* Training ^ 

Techniques/*Supervisory Training/ I 

I 

This volume is largely based on the Conference on Educational Change Agent |~" 
Training which included members from public schools, college and university 
educational departments, educational research centers, and state and federal 

educational agencies. Its purpose is to provide trainers and training program I 

developers with guidelines for effective use of training programs and of ^ 

available resources. The conference members initially discussed various research ! 

findings and theory which they believed to form a sound basis for training models I 

in the change process. Four specific goals of training and fifteen principles of | 

good training design are developed and defined. Based upon these goals and , 

pr'nciples, a comprehensive outline consisting of eight elements is developed j 
and described to serve as an evaluative checklist for training programs. The 

conference members then divided into groups to use the criteria developed to i 

f^:rmulate various types of training program models. The results are presented | 
in outline form and cover the following areas: training school systems to 

develop a self-renewal capacity; producing structual and political changes in ' 

school systems; linking school systems with available resources; and improving ' 

the effectiveness of other educational agencies. Also presented is a fully ' 

developed, ideal model program for state educational agencies to train in ' 

change planning and management skills. (MG) j 
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Training for Leadership in Local Educational Improvement Programs |^ 

Heathers, Glen ' 

1700 Market Street, Philadelphia, Pennsylvania 19103 ' 

Research for Better Schools ' 



*Change Agents/*Dif fusion/*Educational Change/*Educational Innovation/*EdiJcat1onal i 
Iniprovement/*Educational Strategios/*Individual Instruction/*Instruct1onal Materials/. 
^Leadership/*Professional Training/*Professiondl Continuing Education/*Program 

Description/*Program Design/*Program Evaluation/*School Personnel/*Social Change/ ' 

The program trains personnel for a specialty in designing, conducting, evaluating; 

and diffusing a local change program. Ten self-contained modules cover the following ( 
aspects: problem-solving model for designing and conducting local improvement program^, 

working relationships in leadership for change, change theory and strategies, [ 
educational reform themes and related innovations, implementing innovative programs, 

and diffusing innovations within school systems. A module can be studied indepen- ' 

dently of the rest of the program; some modules are prerequisites to others. Each ' 

module contains an explicit statement training situations, and an achievement I 

posttest. Although the modules are self-instructional , it is recommended that an i 

instructor be available to assist trainees with problems encountered and to assess i 
accomplishment of objectives. Each module requires from 6-15 hours to complete and 
can be used for inservice or preservice training. Directions for assessing products 

of trainee practicum exercises are also included. Access to an educational library, I 

ERIC, and other reference sources is important. In addition, access to educational , 

R&D agencies and school systems 1s nocessary for practicum experiences. The target ^ 

audience is designated as school district administrators and specialists and field ^ 

consultants in state education departments, educational labs and centers, schools of ^ 

education, or private agencies. The program will be available from the developer , 

after July 1963 and from a publisher by January 1974 at an estimated cost of $10 , 

or less per module. i 
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How to Lie with Statistics 

Huff, Darren 

New York, New York 

W. W. Norton & Company 

1954 ^ 

♦Mathematical Enrichinent/*Statistical Bias, Sampling/*Statistical Analysis/ 
♦Statistics, Mathematical Applications/*Supplementary Textbooks/*Secondary 
School Mathematics, College Mathematics/ 



This basically non-mathematical book introduces the layman to common 
statistical measurements and technqiues by repeatedly exemplifying their 
use to mislead, confuse, inflate, sensationalize, and oversimplify. A 
distinction is made between mode, median and arithmetic mean; the term 
"average" is dismissed as meani>^sless when used without qualification. 
Stratified, random and representative samples are distinguished and the 
selection of biased samples is explored. Consideration is given to mis- 
leading statistics about distributions when it is not specified whether 
the distribution is normal or skewed. Ranges, degree of significance, 
degree of precision, probable error and standard error are defined; then 
certain data are shown to be of little value without the presence of one 
or more of these measurements. Various deceptions in line, bar, and 
pictorial graphs such as distortion of measurement, divisions, showing only 
part of the total graph, and changing the coordinates of the ordinate and 
abscissa are exemplified. Studies are reported of biases which result 
from improper questionnaire and interview survey techniques. The use of 
deceptive words, linear regression or cyclical variations, and cause- 
effect are also demonstrated. In conclusion, five simple questions are 
formulated which should be posed and answered to avoid being misled 
by statistics. (MG) 
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Ha ndbook 1n Research and Evaluation i 

Isaac, Stephen i 
San Diego, California 92107 i 
Robert Knapp i 

1971 I 

♦Statistical Ana lysls/*Behavl oral Science Research/^Eaucatlonal Research, ~ ^ 

Technical Writing, Educational Researchers/*Research Criteria/*Research • 
Design/*Research Methodology/*Research Projects, Research Proposals, Research I 
Skills, Research Tools/*Measurement Techniques, Evaluation Criteria/*Evaluation i 

Methods/*Evaluation Techniques/ ^ 

This handbook is a collection of principles, strategies and methods which , 
are useful in the planning, design, and evaluation of research in the behavioral 
sciences and education. Using almost a check-list approach, the handbook is ' 
designed to present an overview, a summary of alternatives, or a listing of I 
strengths and weaknesses of various techniques for behavioral research. The I 
volume is intended as a general reference tool for a research team, project I 
director, proposal writer, evaluator, etc. Initially presented are several i 
sets of check-lists and guidelines which should prove useful in research I 
planning. As research design depends upon the purpose and nature of the study, , 
nine design alternatives are briefly described. Also, various possible problem , 
areas which the researcher must consider such as validity, regression effect, 
etc., are detailed. Various techniques for establishing measurable criteria I 
for research are discussed. Bripfly surnnarized next are the more commonly | 
used statistical techniques and guidelines which are provided for their use. 
Finally presented are guidelines for planning, preparing, writing, and ' 
evaluating a research proposal, thesis, report, or article. (MG) ^ 
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Program Evaluators Handbook > Determining Objectives 

Johnson, Mauritz 

Albany, New York 12224 

The State Education Department 

1969, 41 p. 

*Affective Objectives, Cognitive Objectives/'^Educational Objectives , Psychomotor 
Objectives, Program Proposal s/*Eva1uation Criteria/*Program Evaluation, Summative 
Evaluation, Evaluation Methods and Models, Conceptual Schemes/ 



This booklet is concerned with the formulation of project objectives and a j 
general examination of the evaluation process. The material is one of the results 

of review and training sessions held by the State of New York for strengthening ^ 

the evaluative efforts of the Elementary and Secondary Education Act, Title III. I 

The following evaluative models are utilized for a discussion of describing and ' 

appraising educational procedures and results: 1) Stake's evaluation matrix; | 

2} CIPP Model; 3) Curriculum-Instruction Model; 4) Hill's Administration Model; I 

5) Guba and Clark'j Change Model; 6) Gideonse's Change Model; and 7) Stufflebeam's | 

Feedback Control Loop. In addition, 13 evaluative criteria for proposals are ^ 

listed and state educational goals and general educational needs are detailed. | 
If developers have not listed program objectives, the evaluators must do so in 

terms which permit evaluation. There is a general discussion to distinguish I 

goal oriented objectives and societal goals. Taxonomies of objectives are j 
then listed for the cognitive, psychomotor, and affective domains* After briefly 

discussing the qualities of objectives, a list is included of objectives proposed ' 

by various bureaus of the State Educational Department. Finally, practice ' 

exercises are provided for: clarifying objectives; marking inconsistencies; ' 

translating needs into problems; appraising the clarity and achievability of ' 

stated objectives; assessing priorities; and writing objectives in behavioral ' 

terms. (MG) ' 
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Experimental Assessment of an Incentive Program to Enhance School Learning: 

A Pilot Study, Final Report to Franklin-McKinley School Dirtrict 
Jung, Steven, M,, Lipe, Dewey, Canter, Shelly 
P.O. Box 1113 Palo Alto, California 94302 
American Institutes for Research 

July 15. 1971 . 43 p. ■ 

*Motivation, Incentive Systems/*Motivation Techniques, Pilot Projects, 
Cognitive Objectives, Achievement Gains, Classroom Research/*Learning 
Motivat1on/*Student Motivation/*Teacher Motivation/ 



A pilot study to ascertain the effects of utilizing six educational 
incentive models for rewarding? teachers, students, and parents in producing 
student learning gains is reported. The purposes of the pilot study were 
to: determine if a successful experimental field study of incentives 
could be implemented; develop and refine the methodology necessary for 
a large-scale field study; and ascertain preliminary estimates of incentive 
effects for assessing the potentiality of further research on the topic. 
The study specifically examined the effects of incentives in acquiring 
mathematics and reading skills for children, grades 1-3, over a 4-5 week 
period. Three experimental schools, two active control schools, and one 
passive control school were involved in the project. The instructional 
material was individualized for each student and incentives were provided 
if the class as a whole achieved a required standard of mastery. The 
various incentive models, types of incentives, and their effects on student 
performance and student, parent, and teacher attitudes are discussed. 
Two techniques for monitoring the instructional process are described. 
Tabular data results are presented on admihistered pre-, and post-, 
standardized tests, and on criterion-referenced tests. Appendixed are a 
comprehensive list of objectives for primary reading and mathematics 
and various letters, questionnaires, procedures, etc., used for the 
pilot study. (MG) 
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An Introduction to Learning Games & Instructional Simulations, 

A Curriculum Guideline 
Klietsch, Ronald G,, PhD, 

Instructional Simulations & Co, 

iafi3 

^Educational Games/*Game Theory/*Simulation/*System Approach/*Systems Concepts, 
System Development/ 



This non-mathematical, theoretical treatise first examines learning games 
and instructional simulations as systems and then discusses methodological con- 
siderations- The work's aim is to introduce basic ideas and procedures to 
social and behavioral science students. After considering the various concepts 
of systems, their factors, characteristics, and stages are discussed. Several 
action restraints, constraints, and processes of systems are then examined- 
The purposes, operations, forms and fabrication, of models from a simulator's 
perspective are considered. Distinctions are made between real , contrived, and 
quasi simulations and games. The following qualities of simulations are 
discussed: purposes; operational specifications; guidelines for fabrication; 
and learning values. The instructional and learning values, data generating 
capacity, and fabrication of learning games are discussed. Finally, research 
variables and criteria for testing and evaluating, future research needs and 
directions, and model and methods limitations of learning games and instruc- 
tional simulations are considered. (MG) 
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A Programmed Introduction to Educational and Psychological Measurement 
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*Test Construction/*Test Interpretat1on/*Test Reliability, Test Reviews/*Test 
Validity, Supplemental Textbook/*Measurement Techniques, Programmed Instruction/ 
*Correlations , Discriminant Analysis, Predictive Validty, Item Analysis, 
Standard Error of Measurement, True Scores, Behavioral Sciences/ 



Us 1 ng a programmed instruction approach, this volume discusses the basic 
concepts of measurement as they relate to test development and interpretation. 
The work is designed to supplement texts in measurement and evaluation in the 
behavioral sciences and is written for individuals who have had an introductory 
course in statistics. The following topics are discussed: 1) the differences 
between qualitative and quantitative measurement and between discrete and 
continuous variables; 2) nominal, ordinal, interval, and ratio measurements; 
3) adequately defining the phenomenon to be measured; 4) concurrent and 
predictive validity; 5) the Standard Error of Measurement; 6) test reliability 
and methods for determining the degree of stability in test scores over a 
short time period; 7) the use of equivalent tests to determine reliability; 
8) the internal consistency of a test using such measurements as the coefficient 
of stability, the coefficient of equivalence, the coefficient of internal 
consistency, and the Spearman-Brown Prophecy Formula; 9) using a correlation 
matrix to determine which test items to delete or retain; 10) difficulty 
level and discriminating power for item analysis and the rationale and 
techniques for computing the Ease Index and the Discrimination Index; 11) 
ranks and percentiles; 12) test norms; and 13) use and interpretation of 
z scores, t scores, stanines, and deviation IQ's. (MG) 
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Analyzing Performance Problems or 'You Really Uughta Wanna' ~ r 

I 

Mager, Robert F., Pipe, Peter i 
6 Davis Drive, Belmont, California 94002 i 
Fearon Publishers/Lear Siegler, Inc., Education Division i 

1970, 111 p, I 

*Performance/*Performance Factors/*Work Atti tudes. Work Environment, Task Analysis/^ 
*Task Performance, Work Simplification, Personnel Evaluation, Behavior Change, i 
Behavior Patterns/*Failure Factors/*Low Achievement Factors/ t 

I 

This volume presents a detailed procedure tor identifying and analyzing ^ 
the nature, importance, cause, and possible solutions of performance problems, 
i.e., when an individual's actual performance differs from the desired performance.' 

Although the technique is applicable to everyday life, the volume is basically ' 

oriented towards educational and industrial environments, from which numerous ' 
examples are used. Utilizing a flow diagram and a list of appropriate questions, i 

the first steps are to determine the nature and importance of the performance i 

discrepancy and whether it is due to an actual skill deficiency* If a skill | 

deficiency exists, one then determines: 1) whether the skill once existed; ^ 

2) whether the lost of deteriorated skill is used frequently or infrequently; ^ 

3) whether there is a solution simpler than performance maintenance of formal 
training; and 4) whether the person has the potential to perform the skill. I 
If a skill deficiency does not exist, one determines: 1) whether desired | 
performance leads to unfavorable consequences; 2) whether non-performance or . 
other performance leads to more favorable consequences; 3) whether there is a 
meaningful consequence for desired performance; and 4) whether there are ^ 
obstacles to the desired performance. Proposed solutions are presented for i 
all the above possible causes of performance problems. Finally, one must^ ' 
compare the size of the remedy with the size of the solution. In conclusion, ' 
a checklist of key issues and appropriate questions for analyzing performance ^ 
problems is summarized. (MGl 1 
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Preparing Instructional Objectives, (Second uopy - Krepanng Ubjectives 

for Programmed Instruction) 
Mager, Robert F., PhD. 
6 Davis Drive, Belmont, California 94002 
Fearon Publishers/Lear Siegler, Inc., Educational Division 

1962, 60 p. 

*Behavioral Objectives, Cognitive Objectives , Tajcatlona I Objectlves/^Measurement 
Goals/*Tra1ning Objectives/*Course Objectives/*Performance Criteria, Programmed 
Instruction/ 



This volume utilized programmed instruction to explain now to speciry and 
communicate instructional objectives in concrete, evaluative form. It is 
written for teachers and student teachers in all subjects and at all levels 
who are interested in preparing auto-instructional materials. The book's 
objectives are to teach the reader to recognize instructional objectives which 
are stated in terms of performance, to Identify those portions which define 
acceptable performance, and to select from various test Items those which 
appropriately evaluate the stated objectives. The importance and advantages 
of usefully stated objectives are discussed using numerous examples. The 
following qualities of meaningful objectives are described: identifying the 
desired behavior; specifying the conditions under which this behavior should 
occur; and specifying the criteria of minimally acceptable performance. A 
self- test is provided to determine the reader's ability to identify the 
characteristics discussed. Supplemental test is also included but not 
necessary to achieving the document's objectives. There is no discussion of 
the philosophy of programmed Instruction, who should select objectives, or 
which objectives should be selected. (MG) 
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Urban Educational Researchers Training Institute * 

t 

McAdoo, Harriette & John ' 
Washington, C, 20001 . ' 

School of Social Work, Howard University ' 

. J 

*Data Analysis/*Data Collection/*Educational Research/*Evaluation/*Field E)<perience/ j 
*Institute (Training Programs )/*Literature Reviews/*Minority Role/*Professionai Training; 
*Program Description/*Research Design/*Research Methodology/*Research Problems/*Research 

Proposal s/*Research Skills/*School Personnel /technical Reports/ ^ 

< 

The Institute provides an instructional model for use by school systems, community 
agencies, college and universities interested in training research and evaluation | 
personnel involved in urban school settings. Two six-credit courses were designed fojr 
the institute, sponsored by Howard University, They cover research proposal writing,! 
statement of the problem, use of theory in the development of hypothesis, review of 
literature, research design-population selection, instrumentation, data collection antl 
analyses, and report writing. Each course was a semester in length. Readings came t 
from various research books supplied to the trainees. The trainees spent two weeks I 
in classes developing a research proposal related to a problem in their home school i 
systems. They are receiving monthly on-the-joh supervision in carrying out their j 
proposal, followed by two on-campus weekends for progress reporting and individual 
instruction. At the end of the project, they will spend two weeks on campus analyzing 
the data from their research project, writing up the findings, and receiving further | 
training in research evaluation. Ten modules for the training will be available in ^ 
October 1973, 
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Program Evaluators Handbook, Sampling Model 

Meyer, Donald 

Albany, New York 12224 

The State Education Department 

1969. 19 p, , 

*Bayesian Statistics/*Sampling, Statistical Analysis, Statistical Bias, Statistics, 
Evaluation Techniques/ 



A Bayesian model for determining a variable's confidence interval, given i 

there were random and non-random sampling errors, is presented in this booklet. j 

Initially, a standard formula is presented for determining a confidence interval , 

when there is random sampling error. The following types of sampling errors . 

are then discussed: frame error; selection error; non-response error; measure- ' 

ment error; and random sampling errors. Given an example problem with estimated ! 

error values for the above error parameters, a tree diagram is utilized to I 

perform a population analysis and to show the proportion of the various error | 

sources • The possible dependent relationship among the error parameters may i 

be eliminated by the formulation of new parameters which are ratios of the | 
original error parameters. A formula is given for this error ratio model which 

incorporates formulas for frame, selection, random sampling", non-response, and ^ 

measurement errors. The ratio model is then used to perform a total error | 

analysis for an example problem. (MG) i 
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Appraising Educational Research: A Case Study Approach 

Millman, Jason & Gowin, D. Bob 
Ithaca, New York 14840 

Department of Education, Stone Hall, Cornell University 



^Analytical Crit1cism/*Case Studies (Educat-ion)/*Cri tical Reading/*Educational 
Exper1rnents/*Educational Research/*Instructional Material s/*Philosophy/*Prediction/ 
^Research Criteria/*Research Design/*Research Methodology/*Research Skills/*Status/ 
^Theoretical Criticism/*Training Techniques/ 



This workbook is intended for use in improving the ability of learners to apprais 
critically educational research. The print materials consist of the following: an 
introductory statement about the nature of criticism, a statement about the contents 
of the materials and suggestions for use, and nine case studies. The problem areas 
in the nine cases cover: grading and student attitudes, evaluation of a Head 
Start program, prediction of long-term success, reinforcement and behavioral 
modification, effect of questioning procedures on student achievement, development 
of a concept of justice, factors affecting the validity of creativity ossessments, 
experimentation with live animals in elementary school, and small group composition 
and productivity. The types of research in the cases include: status, prediction, 
experimental, case study, and philosophical analysis. Students read each article 
and special notes and then respond to orienting questions either verbally (if in 
a class situation) or in writing. They compare their responses to model appraisals. 
Comprehension of the articles requires neither statistical sophistication nor 
expertise in the substantive field. The target audience is research and research- 
related personnel, including students in research methods courses. The workbook can 
be used in conjunction with a research methods course or separately. It is expected 
to be available in Spring, 1973, at a cost of approximately $5, from Prentice-Hall, 
Inc., Englewood Cliffs, New Jersey. 
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Procedures for Conducting an Apprenticeship work-Study Program 



Millstein, Eugene J., Melnotte, Judith M., Dunn, James A. 

Palo Alto, California 

American Insti lutes for Research 

1972, 0^G:p-71-7412, 87 p, 

Research Skills/*Research ApprenticesFrips/*Training/*Tra1ners/*Trainees , 
Training Objectives, Professional Training/*Training Techniques/*Educational 
Researchers, Educational Research/ 



This manual describes procedures for conducting a cross-training and 
apprenticeship program for the individuals interested in a career in educational 
research and development. It is intended as a guide for those interested in 
administering such a program and is based on the experiences of a pilot program 
conducted by the American Institutes for Research. The program description 
contains information on: length of the program; site; target population; 
instructional system; curriculum; recruitment procedures; selection criteria; 
stipend; and assistance with job placement. The following information on the 
instructional model is then provided: curriculum outline; how to identify 
individual objectives, prescribe learning activities, and monitor trainee 
progress; and how to select, assign, and monitor apprenticeship tasks. 
Publicity, application, and selection procedures, and program management 
considerations are then discussed. The importance of job placement activities 
if the program is used as an independent training program or as an adjunct to 
regular academic training is discussed. Appendixed are a list of proposed 
instructional objectives, a bibliography of instructional materials and 
resources, and a sample fact booklet for prospective applicants. (MG) 
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A Seminar and Training Program in Needs Assessment and Goal 


Development , 


Morgan, James M. 




230 East Ninth Street, Cincinnati, Ohio 4*205 




Cincinnati Public Schools 





* Administrator tducatlon/*tducational Object1ves/*Educat1onal Needs/ ^Community ^ 
Involvement/*Goal Orientation/*Individual Instruction/*Instructional Material s/*Needsl/ 
*Professional Training/*Program Description/*School Personnel/*Seminars/*Simulations/t 
*Tape Recordings/*! ransparencies/ i 



Two sets of complementary materials are provided for t- n'ning on needs assess- 
ment and goal development in local school districts. The first is a Training 
Program for school -community groups on settings goals for their school, based on 
comprehensive needs assessment. It consists of eight modules, accompanied by 
materials for a trainer. The trainer is allowed some flexibility in arranging 
the materials for adaptation to the local situation. The target audience is local 
school staff, parents, and community residents. The second set of materials takes 
the form of a university-level seminar for training professional-level developers, 
administrators, and evaluators in similar sills on a more sophisticated level. 
These materials also have a modular format. Both sets make extensive use of 
simulations, tape-slide presentations, and self-instructional exercises. Self- 
evaluation exercises on the content of each module are provided. Each program takes 
about 16-20 hours to complete, with o e trainer per 12-14 participants. The programs 
will be available in Fall 1973. 
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*Course Organization/*Educational Strategies/*Design/*Field Expertence/*FonDative 
Evaluation/*Participation/*Professional Training/*Program Design/*ProqrajT} Description 
^Program Improvement/*Prograni Planning/*Systems Development/^Transparancies/ 



The program still in field testing, is designed to train educators in the use 
of a strategy, the IDEALS concept, specifically developed for design or development 
work. The IDEALS (Ideal Design of Effective and Logical Systems) concept has four 
parts: (a) a prescriptive, universal framework or system for specifying any 
solution in terms of eight elements in five dimensions; (b) a design strategy of ten 
steps, dealing with functions and ideal system targets; (c) a focus on functions 
and targets to unite organizational personnel in a positive participatory framework 
for continuing system design and improvement program; and (d) use of models only 
when helpful and appropriate. The sequence of activities begins with a three- 
week training course on the IDEALS concept design strategy and related concepts, 
followed by two months of application of the strategy to projects in the partici- 
pants' organizations, a two-day review and question answering workshop for solving 
problems which developed during the application period, and a final three-day 
report and wrap-up workshop for evaluation. The target audience is specified as 
educational professionals in critical decision-making positions at various organiza- 
tional levels. The materials include a basic textbook, Work Design: A Systems 
Concept , as well as transparencies, slides, and overhead projector. An instructor 
with a background in system design is needed for the three-week course. 
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Basic Program Plan, Accountable Learning Systems 



Durham, North Carolina 27701 ' 

National Laboratory for Higher Education ' 

Anril K IQ??. 0EC-2- 7-Q62556-30 7q . 81 p. ' 

*Junior Colleges, Community Colleges, Instructional Improvement/*Ef f ecti ve ^ 

Teaching, Educational Objectives/*Educational Accountability, Educational ' 

Development, Educational Programs, Professional Training, Training Techniques/ ! 

♦Training Objectives/ i 

L 

The basic components and evaluation strategies of the Accountable Learning t 

Systems (ALS) program, which is designed to increase instructional effectiveness j 
and relevance in two year colleges, are described in this volume. The overall 

objective of the program is to develop tested products for training faculty I 

and administrators in clarifying college goals and objectives so that ' 

individualized courses of instruction may be systematically developed. The I 

program description discusses: 1) significance of the problem to the target I 

audience; 2) expected outcomes; 3) assumptions; 4) overall strategy; and I 

5) program implementation. Numerous product descriptions and schedules i 

are described for the following program components: 1) training for instruc- | 
tional accountability; 2) management support for instructional accountability; 

3) curriculum development for instructional accountability; and 4) technical ^ 
assistance and service. Extensive evaluation charts, which use a criterion- | 
referenced approach, are included for determining the efficiency and effective- 
ness of the ALS program. These illustrate several evaluation strategies and 

modes of data collection and reflect the current status of evaluation plans 

and results. The charts contain the following information, when available: 

1) identification of the program aspect or product being evaluated; 2) | 

questions the evaluation is designed to answer; 3) performance criteria; 

4) the design, data collection, and analyses methods utilized; and 5) test ; 
results or the schedule dates for testing. (MG) ! 
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Practical Programming 
Pipe, Peter 

Holt, Rinehart and Winston, Inc, 

1966. 70 p, ^ ^ 

*Programming/*Programmed Instruction, Programmed Ma"terials/*Programmers/ 



♦Programming Methods, Evaluation Methods/*Branching/*Sequential Programs, 
Constructed Response, Fixed Response/ 



This volume is designed to help start the beginner in writing a program; 
it covers practical issues such as preparatory steps, programming techniques, 
and testing and analysis methods. It is written for anyone who has decided 
that he is going to write a program. A brief introduction to programmed 
instruction defines four of its major characteristics. Also discussed are 
the characteristics, advantages and disadvantages of the two major programming 
approaches, linear programming and branching programming. Six steps necessary 
for program writing preparation are detailed and discussed. In actually 
writing a program, its general format and such issues as Introduction content, 
amount of practice necessary, and when to summarize are discussed. The 
written program should contain the following discussed components: 1) 
introduction; 2) review of essential concepts assumed as prerequisites; 3) 
step-by-step development of new concepts; 4) practice; and 5) final summary 
and criterion test. Numerous techniques and considerations in designing 
branching and linear programs are discussed using diagrams and examples. 
Finally, techniques for program testing, revision, and editing are briefly 
described. (MG) 
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An Evaluation Guidebook, A Set of Practical Guidelines for the Educational 

Evaluator 
Popham, W. James 

Box 24095, Los Angeles, California 90024 
The Instructional Objectives Exchange 

1972. 89 p. 

*Evaluation Criteria/*Evaluation Methods/*Program Evaluation/*Evaluation 
Techniques/*Educational Objectives, Performance Criteria, Performance 
Specif ications/*Performance Tests, Measurement Goals/*Measurement Techniques/ 
*Data Analysis, Data Collection/ 



This volume brings together and describes a collection of relevant, 
recently developed, technical procedures which may be used as a practical 
set of guidelines for the educational evaluator. After defining educational 
evaluation, its two main roles are described: identifying the instructional 
objectives and judging the quality of the instructional procedures designed 
to attain the objectives. The guidelines for instructional objectives 
concern: 1) the role of measurabili ty; 2) unmeasurable objectives; 3) 
acceptable perfonnance criteria; 4) content general vs. test item equivalent 
objectives; 5) minimal proficiency levels; 6) taxonomic analysis of objectives; 
and 7) utilizing existing collections of objectives* Guidelines for measure- 
ment examine: 1) criterion-referenced measures; 2) domain-referenced achieve- 
ment testing; 3) multiple criterion measures; 4) unobtrusive measures; and 
5) unanticipated outcomes. The guidelines for data collection and analysis 
concern: 1) clarifying value preferences; 2) compeHng preference and 
performance data; 3j person and item sampling; 4) formative evaluation; 
5) summative evaluation; 6) appropriate units for data analysis; 7) estimation 
vs. hypothesis testing; and 8) cost/effectiveness decision making. Each of the 
above guidelines suggest a practical course of action for the educational 
evaluator. Finally, an example is presented of an evaluator who utilizes 
the guidelines. (GS) 
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Improving the Program, Revising Instructional Materials in Light or 

Tryout Data, Booklet 5 
Rosen, Marvin J. 
Durham, North Carolina 27701 
National Laboratory for Higher Education 

197K 29 p. 

♦Performance Criteria/*Perfonnance Factors, Educational Problems, Evaluation 
Techniques, Evaluation Methods, Student Evaluation/*Evaluation Thinking, 
Formative Evaluation/*Failure Factors/*Low Achievement Factors/ 



This booklet describes a procedure tor analyzing student responses to 
identify the types of deficiences in an instructional program and to determine 
specific, remedial revisions. Student responses are to be obtained from 
criterion-referenced tests, practice exercises, and interviews. To locate the 
program deficiences, a flow diagram for their analysis details all the steps 
which should be examined before determining a possible program revision. 
First, the defective element should be identified and determine whether it is 
sufficiently important to warrant a revision effort. If it is not very 
important, the criterion for acceptable performance might be lowered or the 
associated instructional materials could be eliminated from the program. 
If the material is important but the student is not performing the learning 
activities associated with it, it should be determined whether: 1) the students 
know what they are supposed to do; 2) they are trying; 3) they have the materials 
necessary; and 4) there are other obstacles to performance. If the student 
is performing the learning activities, but unsuccessfully, it should be 
determined if the criterion measures are valid and if the students are making 
errors when practicing. For each of the above possible problem areas, remedies 
for their solution are briefly discussed. (M6) 
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What Are S':u dents Learning?, Documenting Multiple Effects of instruction 

on Learners, Booklet 1 
Rosen, Marvin J. 
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January. 1972, 44 p, 

*Objectives, Affective Objectives, Cognitive Objectives, Psychomotor Objectives/ 
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This booklet describes a series of procedures tor specitymg concrete 
objectives and briefly discusses the measurement process. The following 
three types of information in a well -formed objective are described: 1) 
outcome formation, or the behavior desired; 2) level of achievement for- 
mation, or the specification of minimal acceptable performance; and 3) 
circumstances of evaluation formation, or conditions under which performance 
will be evaluated. The differences between general and spedific objectives 
and refining the general objectives into specific objectives are discussed- 
A form for refining general objectives so that they clearly and unambiguously 
define the skills, knowledges, and attitudes to be acquired is provided* 
The Three major classes of learning, psychomotor, cognitive, and affective, 
are defined and exemplified. Methods are discussed for identifying and 
classifying these types of learning by level of difficulty or complexity, 
Finally, the necessity of specific procedures for verifying, testing, ob- 
serving, or noting student achievement of specific objectives is briefly 
discussed. (MG) 
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Which Students Achieved Which Objectives? Summary Statistics for 

Documenting Criterion-Referenced Instruction, Booklet 2 
Rosen Marvin J. 
Durham, North Carolina 27701 
National Laboratory for Higher Education 

January, 1972. 23 p. 

^Criterion-Referenced Tests, Performance Criteria, Evaluation Criteria, 
Date Analysis/*Test Interpretation/*Test Results, Program Effectiveness, 
Evalution Techniques/ 



This booklet details a procedure for obtaining evaluation statistics on 
criterion-referenced instruction, which is instruction designed to accomplish 
certain specific objectives. The procedure helps to describe clearly the skills 
mastered and missed by students. The reader should learn to perform the 
following tasks: 1) given one or more test items for an objective and a 
criterion of successful performance, calculate the percentage of students 
achieving the criterion; 2) given objectives with varying criteria for 
successful performance, calculate the percentage of students achieving 
the criterion for each objective; and 3) given criterion levels of success- 
ful performance for objectives and the precentage of students achieving 
these levels before and after instruction, display in tabular form any dis- 
crepancies and the effectiveness of the program relative to each objective. 
A form is provided for tabulating the achievement of learning criteria which 
is useful in pretests to determine: to exclude individuals from doing an 
instructional sequence for a particular objective if their scores indicate 
that they have achieved the criterion; and areas to be emphasized for those 
who fail to achieve the criterion. It is useful in posttests to determine: 
the students who achieve the criterion after, but not prior, to instruction; 
and if revision of the instructional sequence is necessary. (MG) 
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How To Write A Program, a Programmed Course Designed to Teach Techniques 

for Programming Instruction 
Silverman, Robert E. 
Box 156, Carlisle, Mass. 
Carlisle Publishers, Inc. 

1970 ^ 

*Programming/*Programmed Materia I s/*Programmed Instruction, Auto-Instructional 
Aids, Branching, Constructed Response/*Progranme 's/*Programming Methods, 
Auto-Instructional Methods, Auto-Instructional Programs, Prompting, Feedback/ 
♦Programming Problems, Instructional Technology/ 



This programmed instruction text is an untheoretical introduction to how 
to write programmed materials. Designed for teachers or trainers with no 
knowledge of programmed instruction, the reader learns to apply principles 
derived from the psychology of learning and programming experience. The 
text's objectives are to teach the following: 1) to state objectives in 
terms of desired student responses; 2) to use prompts and proper sequence 
to obtain the desired responses; 3) to use positive, immediate feedback; 
4) to require relevant and effective responses; 5) to use RULEG, E6RUL, 
discrimination, and response-familiarization sequences; 6) to write teach- 
ing, practice, review, and test frames; and 7) to use branching, a technique 
which provides individualization within a program. Instruction and practice 
are provided for each of these basic programming skills and techniques. In 
conclusion, an exercise is provided which is designed to utilize the reader's 
newly-learned skills by actually writing a programmed lesson. The text also 
includes a glossary of technical terms and a bibliography of suggested 



readings. 
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Preparing Program Objectives. Proposal Guidelines for Categorically 
Funded Programs 

Albany, New York 12224 

The State Education Department, Bureau of Urban & Com^nunity Programs Eval. 

April, 1971, 12 p. - 

♦Educational Objectives, Behavioral Objectives, Cognitive Objectives, 
Performance Specif ications/*Perfonnance Cri teria/^Measurement Goals/*Evaluation 
Criteria, Evaluation Methods/*Program Evaluation, Evaluative Thinking/ 



This brief set of guidelines, prepared by the New York State Education 
Department for categorically funded programs, describes techniques for 
writing instructional objectives and the planning of evaluation techniques. 
It is designed as a brief reference tool for local project directors and 
for individuals with little, if any, training in research and evaluation 
design. To obtain a meaningful evaluation of a program, it must be well 
designed and have specific objectives which are measurable. After des- 
cribing the characteristics of broad program objectives, the following 
components of specific objectives are discussed: describing the educational 
intent; describing terminal performance by identification of the performance 
conditions under which it should occur, and defining a standard for 
acceptable performance; providing a separate statement for each objective; and 
specifying n^ore than one statment to increase the probability of communicating 
its intent. A four question test on performance objectives is provided for 
analyzing proposed objectives to determine the adequacy of their description. 
In planning for program evaluation, the following evaluation procedure com- 
ponents are described: determining criteria for program success; planning 
the measures which are to be used; and revealing the results in terms of 
the degree to which the program's objectives are achieved. (MG) 
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Recomnenda tions for Curriculum and Instructional Materials 



Tyler, Louise L., Klein, M. Frances, Michael, William B. 

2265 Prosser Avenue, Los Angeles, California i 
Tyl Press , 
197K IM^ ^_ 

♦Instructional Materials , Instructional^ds7*Se1ection/*rvaTuatibn Xriter^i^ 
♦Curriculum Evaluation, Curriculum Design/ « 



Twen ty- e i gh t recommendations fo r identifying significant aspec ts ot " 
instructional materials are proposed as possible criteria for determining 
the materials* quality and for provding a basis for making judgments regarding 
them. These recommendations are intended for producers of curricula and 
instructional materials, for users in their selection, and for funding agencies 
in their evaluation. The recommendations concern the following aspects of 
materials: rationale, appropriateness, specifications, effectiveness, conditions, 
practicality, and dissemination. After stating the recommendation, it is 
designated as essential, very desirable, or desirable. Each recoiranendation is 
then discussed in relation to several of the following terms: rationality, 
values, decisions, accountability, significance, comprehensiveness, causation, 
behavioral ism, and knowledge. The recommendations are then analyzed and 
evaluated within the framework of a specific teaching model, first proposed 
by Glaser, which consists of four major components: objectives; entering 
behavior; instructional procedures; and the evaluation of outcomes. The 
evaluation of materials and the components of the evaluation process are 
discussed in regard to determining whether the recommendations have been 
utilized. Finally, a transcript of a meeting with the authors and a group 
of teachers concerned with selection of materials is included, (MG) 
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Administrator's Manual for State Plan Programs, Title III, ' 

Elementary and Secondary Education Act ' 
U. S. Department of Health, Education, and Welfare 

Office of Education, Projects to Advance Creativity in Education ' 

1971 ^ „ _ : 

♦Administrator Guides/*Supplefnentary Educational Ce iters7*Program Administration/ 

♦State Departments of Education/*Stdte Federal Surport/*Program Planning » ' 

Educational Change, Educational Planning, Educat-^onal Finance, State Programs, » 

Program Proposals/ i 

This manual attOTpts to provide a base of common understand! ng^amoiTg » 

State educational agencies on the legislative changes occurring in the Title j 
III Program since 1969. Changes in Title III are listed and explained. 

The main text of the manual centers on the details of state plan admiristration. I 
Detailed information is provided on eligibility for Federal funds. State 

organization and functions (State advisory council and State agency), and ' 

the State plan program. Discussion on the State plan program covers the i 

details of the State educational agency as facilitator of educational change, \ 

assessment of educational needs, project development and proposal review, , 

evaluation of projects, dissemination and change strategies, guidance, counseling , 
and testing, other program considerations such as handicapped, hearings, private 

school participation, authority and responsibility, and finance management. ^ 

Other chapters of the manual discuss the purpose of Title III, details of I 
local program development and operation, funding and fiscal considerations, 

and reporting requirements for State educational agencies. An annual report [ 
is presented covering areas and scope of activities to report, dissemination 

activities, instructions for completing the annual report and preparing plans, ' 

certification forms, copyright guidelines, grant terms and conditions, a | 

bibliography, and UCLA's Center for the Study of Evaluation hierarchical | 

objectives charts. (GS) , 
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Categorically Aided Programs "^valuation, Continual Evaluation of Proposals 

and Projects ' 

The University of the State of New York ' 

Albany, New Vork 12224 ^ 

The State Education OepartJTient | 

^Educational Programs/^Evaluation Criteria, Formative Evaluation, Summative ^ 

Evaluation/*Evaluation Methods^ Prograrr Eval jation/*Evaluation Techniques, ' 

Program rroposdls/*Stdte Proqrans/^State Federal Aid/ • 



The evaluation of the State of New York's categorically aided programs i 

at several time points, from proposal submission to product evaluation, 1 

is briefly described in this booklet. Utilizing the CIPP model, which was , 

first developed by Stuff lebeam, the objectives for the following evaluation r 
components are listed: context; input; process; and product. The proposal 

elements which are examined for evaluation purposes are listed and the ^ 

qualities and characteristics of worthwhile proposals are enumerated. ' 

Techniques for monitoring an on-going project are detailed and the I 

characteristics of a worthwhile on-going project are listed. The two major i 

techniques for final program evaluation are cited and the qualities of an i 
exemplary project which merits dissemination are detailed. (MG) 
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Closing The Gap, a Report of the First Two Years of Experience with ESEA, 

Title 1 in New York State 
The University of the State of New York 
Albany, New York 12224 
The State Education Department 

Augusts 1968 , 34 p. 

*State Federal Aid/*State Programs, Educational Programs/*Educationally 
Disadvantaged/*Disadvantaged Youth, Achievement Gains/*State Surveys, 
Reading Improvement/*Statistical Surveys/ 



The major program results of the State of New York's first two years 
experience with the Elementary and Education Act, Title I funding are 
summarized in this booklet. This funding was provided in 1965-67 for 
special educational programs designed to broaden and strengthen education 
for deprived students. Tabular results for the two year program for 
children participating in the following programs and their level of 
improvement are provided: reading programs; pupil personnel services; 
English language arts; and mathematics programs. Selected results for the 
first year are provided on the following programs: grade 4 gains in word 
knowledge and discrimination, reading, and arithmetic concepts; and 
remedial reading for low achievers in high school. Program statistics 
and selected results for the second funded year are provided on: 1) 
attitude changes in grades 1-8; 2) types of programs funded; 3) methods 
used for achieving program goals; 4) type of measuring devices used; 5) 
reading programs by grade level; 6) types of evaluation used; and 7) 
effectiveness of reading programs. Four specific programs are described: 

1) a corrective reading program with a cultural enrichment program; 

2) preparing high school students for college; 3) a pupil transfer program; 
and 4) a comprehensive reading program for grades 1-12. Conclusions and 
implications derived from the program are then summarized. (MG) 
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Urban Education Programs, Evaluation Manual 

The University of the State of Nev/ York 

Albany. New York 12224 

The State Education Department 

June, 1Q69> 68 p, 

*Urban Education, Annotated Bibliography, Measurement Techniques/*Tests/ 
*Performance Tests/*Program Evaluation/*Evaluation Techniques, Evaluation 
Methods, Formative Evaluation/ 



This manual is concerned with the evaluation of projects in the State of i 

New York*s Urban Education Programs, which has two priority areas: 1) projects | 

relating to the development and operation of community education centers; and , 

2) projects directly related to the regular elementary and secondary school ■ 
programs. The manual is designed to aid school districts in developing and 

implementing effective evaluation programs for these projects. The following ! 
basic components of the evaluation plan are discussed: environmental conditions; ' 

objectives; project activities; evaluation design; and decision processes. I 

Model evaluation plans are exemplified in the following areas: school i 

readiness; reading achievement; mathematics achievement; guidance services; i 
retention of dropouts; and effects of cultural deprivation. Each plan states 

its general goal and specific objectives and each objective lists the target ' 

goal performance and evaluation procedures. The following information are I 

provided on various types of evaluation instruments for target group per- , 
formance: description; when they may be used advantageously; types of 
information they should supply; brief guidelines for their design; and good 

and poor examples. An outline which should be used in the development of | 

project evaluation reports is detailed. Finally, appendixed are a list of J 

standardized measurements for basic skills education, samples of evaluation , 

instruments, and an annotated bibliography of selected references in ^ 

evaluation. (MG) . 
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Evaluators Manual for the Evaluation of Federally Supported Research and Development 
Products, (Draft) 

I Palo Alto, California 
American Institutes for Research 

_197?. 18 p . 

*Evaluation/*Summative Evaluation/*Program Evaluation/*Program Effectiveness/ 
^Evaluation Techniques/*Evaluation Methods/*Evaluation Criteria/*Performance 
Criteria, Criteria, Research and Development Centers/ 



This report proposes an evaluation technique for federally funded knowledge 
products and developmental products generated by Regional Educational Laboratories 
and university Research and Development Centers- The manual's purposes are to: 
orient the evaluator; describe the evaluation procedure; provide general in- 
structions; detail the evaluation criteria, and provide information on the center 
and laboratory movement. The following evaluation criteria for development 
products are thoroughly defined: (1) importance of the general problem; (2) 
relevance of the product to th<? general problem; (3) comprehensiveness of the 
product to the problem solution; (4) content clarity and accuracy; (5) 
effectiveness; (6) reasonable cost to carry out and operate the product; and 
(7) market, marketability, and impact potential. The criteria defined for 
knowledge products include #1-3 above and literature discussion quality, 
product originality, research design t^dequacy, interpretation appropriateness, 
justifiable conclusions and recommendations, presentation clarity, and 
potential market. Included is a proposed development product evaluation form 
which contains instructions for its use, abbreviated definitions of the criteria, 
and five briefly stated possible responses for each criteria. (MG) 
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Training Materials for Research, Development and Diffusion Training Programs 

Cuba, Egon G., o.nd Gephart, William J. 

U. S. Department of Health, Education and Welfare, Office of Education 

1970. 767 p. 

*Prnfessional Training/*Material Development, Measurement Techniques, Research 
Design, Research Methodology, Diffusion, Developmental Tasks, Evaluation 
Methods, Evaluation Techniques, Statistical Analysis, Questionnaires/ 



This volume is the result of a project which identified and collected 
semi -developed materials and processes that would be useful for educational 
training programs in research, development, diffusion, and evaluation. 
Materials were identified through: 1) reviewing the literature; 2) indirect 
contact via newsletters, announcements at meetings, etc.; and 3) personal 
contact with individuals involved in research instruction. The 327 partially 
developed materials concern a wide range of research topics; but the majority 
were on statistics, measurement, and research process in general, and research 
design. The volume contains the following information on each of the materials: 
author; address; title; topical focus; purpose; physical description; restrictions 
to use; and general character. Many of the documents are dependent upon other 
printed material or oral presentations; conceptual criteria is lacking for 
judging quality; and student entry, or terminal, behaviors and evaluative 
data are lacking. In addition, the project was concerned with the possible 
refinement, development, and adaption for use of these semi-developed 
materials. Four instructional materials were refined as exemplars: 1) ^ 
coordinated set of transparencies and text on the concepts and activities 
in evaluation; 2) a manual and practice materials for identifying and des- 
cribing segments of the school day; 3) a description of the elements in 
questionnaire construction and an illustratory case history; and 4) a set 
of flow charts and instructional materials for assessing the methodological 
adequacy of completed research. These four developed materials are contained 
in their entirety in the appendix. (MG) 
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Educational Product Evaluation: A Prototype Format Applied 

Glass, Gene V. 
University of Colorado 
Laboratory of Educational Research 

(Educational Researcher). January, 1972. 7 p, 

♦Magnetic Tape Cassettes/*Learning Modalities/*Evaluation Methods/Evaluation, 
Audiovisual Aids, Cost Effectiveness, Master Tapes (Audio), Instructional 
Technology/ 



A format for educational product evaluation is presented and applied to an 
instructional cassette recording. The format, which is adapted from procedures 
for evaluating consumer products, comprises ten steps: product description; 
goals evaluation (what the product's goals are and whether they are worthwhile); 
clarification of point of entry of the evaluator (what choices the product's 
sponsor, creator, and vendor have already made and what choices are still 
open); trade-offs; comparative cost analysis; intrinsic or secondary evaluation 
(technical quality, content evaluation, utilization of uniqueness of medium, and 
importance of medium's unique qualities); outcome of primary evaluation (how 
well the product fulfills its educational goals); summative judgments and 
recommendations to purchasers, sponsor, and vendor; circumstances modifying 
the summative judgments; and evaluating the motive and methods of the evaluator. 

Research findings on aural vs< visual learning are discussed. It is con- 
cluded that a typescript would be more cost-effective than this cassette tape 
for most individual learners, though perhaps not for classroom learning. 

In a response to the evaluation, the developer of the tape, Michael 
Scriven, University of California, Berkeley, in "Letters," Educational 
Researcher, May 1972, explained how the tape was produced. He approved of 
Glass' general procedure, but suggests that the basis for evaluation of 
the tape should have been a comparison with other possible tapes, not with 
a typescript. (AS) 
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Selection and Description of Educational Products for a Study of the ' 

Development and Impact of Such Products i 

Crawfordj Jack, Kratochvil, Dan, Wright, Calvin, & Thompson, Lorna » 

Palo Alto, California i | 

American Institutes for Research i 

April. 197K OEC-0-70-4892, 84 p. ' 

*Criter1a/*Curriculum Design/*Curriculum Development/*Developn)ent/*Diffus1on/*Evaluat1on/ 
^Evaluation Cri teria/*Information Dissemination/*Instructional Design/*Instructional i 
Material s/*Marketing/*Materials Development/^Sunmative Evaluation/ | 

The Evaluation of the Impact of Educational Research and Development Products [ 
Study was to examine the factors in the process of development of relatively . 
successful educational products. Twenty exemplary educationaV products with 
successful impact in schools were to be identified and subjected to an intensive I 
historical review. A major objective of the study is to obtain empirical data ' 
regarding development and diffusion factors which are related to successful product ! 
impact. I 

In this Interim Report, two of the major activities of the study are described | 
in detail. These include the development of initial selection criteria to identify , 
suitable educational products and the identification and description of those producjbs 
meeting the initial selection criteria. 

In the development of the criteria, considerations were given to demonstrated I 
effectiveness in improving student performance; recognition of affective as well as ( 
cognitive goals; products which may contribute to student development by modifying = 
environment, human interactions, or skills; scope of use; definitional criteria for ' 
the purpose of study delimitation. ' 

To identify potential suitable products, the search procedures included: (1) ^ 
literature reviews utilizing ERIC, PACE, and Science Information Exchange sources; ' 
(2) review of selected USOE records not available through standard channels; (3) ' 
a review of previous and current projects which examine the impact of various 
educational programs; and {4) the use of selected nominating panelists representing [ 
various sectors of the educational professions. 

Initial application of these search procedures yielded over 1 ,000 leads to productis. 
Subsequent application of the criteria reduced this pool to 117 products which are < 
described in terms of selection criteria and product characteristics. Comparisons \ 
of product characteristics are summarized in frequency and percentage tables. i 

The relative effectiveness of the procedures used are summarized and recornmendatioms 
made. > 
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A Proposal to Design and Field-Test a Specialized Program Model 

for Educational Planning, Management, Research and Development 
Barus, David N. 

225 West State Street, Trenton, N.J. 08625 

New Jersey State Dept of Education, Div of Research, Planning and Evaluation 



♦Decision Making Skills/*Educational Development/*Educational Planning/*Educational 
Policy/*Educational Research/*Field Experfence/*Management Education/*Planning/*Poli<!:y/ 

♦Policy Formation/*Professional Training/*Program Description/*Seminars/ I 

I 

The New Jersey State Department of Education, a leader in the movement to [ 

advance educational planning, management and research and development, proposes . 
to provide after four months for detailed design, a 12-month full-time intensive 

pilot training program beyond the master's degree level, uniquely combining 1 

theory and practical application. The trainees will be a group of 15 especially ' 

talented individuals carefully selected from a variety of educational, industrial ! 

and other professional backgrounds and academic disciplines, who seek to learn I 

and apply the highly specialized skills, techniques, strategies and overall I 

approaches to policy development and decision-making essential in professional , 

educational planning. This is a field in which there is increasingly urgent | 
need for qualified practitioners at both state and local levels, because of the 

growing diversity and complexity of present educational needs, and the programs I 

designed to meet them, in a time of mounting strains on financial resources and \ 

public insistence on efficiency, productivity and accountability. The project : 
will be carried out in close cooperation with a leading area university, which 

will provide the bulk of the theoretical part of the training in new and existing ' 

courses and special seminars and give appropriate academic recognition of the ' 

trainee's project work. Also collaborating will be one of New Jersey's outstand- ' 

ing local school districts, which, along with the applicant Department itself, will | 
provide the principal field laboratory for a sequence of individually designed 
controlled planning experiences. Formative evaluation procedures will be designed 
for application at each stage as the program activities progress, to assist an 

optimum training experience for each trainee. The final two-month period will ' 

be used to complete the evaluation procedure and prepare the final report ' 

designed for other institutions and agencies considering adoption of the project i 

model and for plans to continue the project. i 
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The Evaluator Development- Program, Parts 1, ii & u, working Paper 
No. 10 

Bates, D. , Buser, R. R., Ellis, J., Rice, D. 

Cooperative Educational Research Laboratory, Inc. 
June 15, 1967, 107 p. 

^Evaluation, Evaluation Needs, Evaluation Methods/*Admimstrator tducation/ 
^Curriculum Development/*Administrator Role/ 



This progress report comprises three parts. 

The first part surveys the expressed need for educational evaluation and 
outlines the evaluator development program. The program aims to promote 
the institutionalization of internal evaluator roles by identifying and 
explicating evaluative tasks, developing and testing needed curriculum packages, 
and formulating evaluator roles. The program is planned as two major projects: 
the ''task development project" and the "position development project." The 
first part of the report details the workplans. 

The second part publishes the program's working glossary of 43 terms used 
in evaluation. Then it presents a list of 44 evaluative tasks, formulated by 
the project staff and based on the literature. The tasks are elaborated and 
their outcomes specified. Tasks range from identifying the goals of an 
institution to helping educators adopt goal-oriented practices. 

The third part explains that position development must await progress on 
task development, but it summarizes the staff's thinking to date and illustrates 
two ways of grouping the 44 tasks into roles: by function (assessment, coun- 
selling, etc.) and by client (teachers or administrators.) (AS) 
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One Year Program to Train Developers in Public Education Systems 
Brown, Roscoe C. 

10 Washington Place, Suite 500, N.Y., N.Y. 10003 
New York University, Institute of Afro-American Affairs > 
December. 1971 



I 



^Educational Development/*Educational Research/*Fic'. J txperience/*Minority Role/ 
^Professional Training/*Program Description/*School Personnel/ 



This program will be a one year, part-time program to train educational i 

developers for public school systems. The primary objective of the program is | 
to train qualified persons to interpret and implement ideas derived from 

educational research in the public schools. Fifteen participants will earn 20 I 

college credits in this program of part-time formal course work and field ^ 

experience. The minimum entrance requirement for the program will be a i 

Bachelor's degree plus at least two years of teaching experience. Participants l 

will be recruited from the New York City and surrounding areas. Special efforts \ 

will be made to recruit Black and Puerto Rican participants and to ensure an , 

equal distribution of males and females. i 

The academic structure of the program is designed to lead the participants 

through the acquisition of skills in a logical progression. Participants ^ 

will be placed in school systems to function as educational developers as a I 
field experience under the supervision of the program staff. The combination 
of formal academic training and field experience is designed to encourage 
a synthesis of knowledge and practical application. 
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Supplementary Proposal for Research Internships in Education 
Cohen, David K. 

24 Garden Street. Cambridge, Mass. 02138 

Harvard University, School of Education, Center for Educational Policy Research 



*Data Analysis/*Educational Pol icy/*Educational Research/*Evaluation/*Field Experie 
♦Institutes (Training Programs )/*Pol icy/*Professional Training/*Program Description/ 
♦Program Proposal s/*Research Methodology/ 



pee/ 



This is a proposal for the development of a new training model for Educational 
R&D personnel, which involves both research internships for second and third year 
doctoral students and one-year institutes for experienced educational personnel. 
Research internships are conceived as a way of offering instruction in the conduct 
of applied research, and the institute as a way of both raising the quality of 
existing staff and improving the character and diversity of training experience. 

New curriculum will be developed for this model in four areas which are seen 
as crucial to improved training for educational R&D practitioners: the effects 
of schooling; the problems of learning from evaluation; data analysis and 
methodological problems in educational research; alternative approaches to policy 
analysis. 

The finished product of the development and pilot project will be: a complete 
description of the training model, and evaluation of its pilot test, and complete 
curriculum materials for the four training areas: these would be available in 
August of 1973. 



Harvard University, School of Education 
Center for Educational Policy Research 
24 Garden Street 
Cambridge, Mass. 02138 



ERIC 



, FWL 027 0246 

Technical Paper No. 1: Procedures Used in Designing a Training 
Program in Educational Evaluation and Development 

University of Colorado, Boulder, Colorado 80302 

Colorado Center for Training in Educational Eval- & Development 

OEC-Q-70>4770 (520), U. S. Office of Education, 15 p. 

♦Educational Development/^Educational Programs/*Evaluation/*Educational 
Research/ 



This paper outlines the procedures ussd in designing a training program in 
educational evaluation, and development. Data were gathered from the following 
sources: 1) existing manpower studies; 2) informational inputs from the U. S. 
Office of Education or its projects, (a) the original RFP 70-12, (b) briefings 
with Office of Education representatives, (c) the AERA Task Force on Research 
Training, (d) the Teaching Research Project to generate information to support 
planning for training in research, development, diffusion, and evaluation; 
3) group and individual meetings with consortium units; 4) working papers drafted 
by several of the consortium units; 5) questionnaires sent to agencies nation- 
wide and within the Rocky Mountain area; 6) staff meetings at Colorado; and 
7) projected conference on educational evaluation and development. Sections on 
each source provide more detailed descriptions of needs, directions, and ob- 
jectives of the training program under consideration. The questionnaires of 
the Colorado Center for Training in Educational Evaluation and Development are 
appended to the paper. This is the first of four technical papers describing the 
CCTEED training program design. The manpower studies indicated that the number 
of personnel being trained for educational research and development was in- 
sufficient for current needs, and that most research training was conclusion- 
oriented rather than decision-oriented. The RFP emphasized the need for 
training educational developers, diffusers, and evaluators; for retraining 
personnel already on the job, filling roles in these areas; and for training 
personnel below the level of the independent investigator (i.e., support 
personnel). The CCTEED training program concentrates on evaluation almost 
exclusively. Questionnaires were sent to all R&D centers, all regional 
laboratories, the thirty largest school districts in the U. S., and to selected 
research projects, agencies, and organizations who were thought to have personnel 
needs in evaluation and development. Also, smaller school districts and moderate 
school districts within the regional consortia were sampled. (MB) 
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A Model Pilot Program for Training Personnel to Develop Solutions 

to Major Educational Problems 
Cullinan, P. A., Merrifield, P. R, 
4 Washington Place, Room 278, N.Y., N.Y. 10003 
New York University, Divs. of Behavioral Sciences & Ed. Admin. 
January, 1972 



♦Educational Problems/*Field Experience/*Individual Instruct! on/*Institutes (Traininc| 
Programs )/*Instructional Material s/*Professional Training/*Program Description/*Resea|rch 
Design/*Research Problems/*Research Projects/*Research Skins/*Research Utilization/ 
♦School Personnel/*Seminars/*Tape Recordings/*Videotape Recordings/ 



AR&D training model emphasizing recruitment of indigenous personnel from com- 
munity school districts for training in operational settings. Research skill 
development based on local educational problems identified by trainees. Summer 
institute followed by two afternoons per week during academic year with staff 
and trainees meeting in school districts for seminars and mini-courses developed 
to meet needs of research problems to be solved. Extensive individual guidance 
in situ by training staff including review of video and audio records of meetings 
and participant presentations. Objectives include definitional, conceptual, design, 
analytical, and interpretive skills. Four-phase sequence from introduction 
(institute) to completion and reporting research study of local educational 
problems. Non-conventional approach to learning research skills. Recruitment 
priorities for women and minority group members. Formative evaluation in course 
provided by frequent checks on mastery of research skills and conferences with 
district and agency management on direction of program. Summative evaluation of 
trainees will be based on the nature and quality of the individual research pro- 
jects, including presentation and dissemination of results to district staff 
personnel, school board, and community organizations* University developed 
product will be a descriptive report on arrangements, training procedures, and 
resultant instructional modules in the research skill areas. These should prove 
useful for adoption by other universities, R&D centers, regional units such as 
Boards of Cooperative Educational Services. Major input to formative and summative 
evaluation of total project will be made by independent outside evaluatpr. Project 
is to be clearly coordinated in tandem with "developer" training by other unit 
in University. 
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Academic Year Institute: R&D Personnel Training Program 
Gunnel 1 , James B. 

29 W, Woodruff Ave., Columbus, Ohio 43210 

The Ohio State University, College of Ed, Faculty of Ed Development 
December, 1971 



*Educational Development/*Educationa1 Research/yinstitutes (Training Programs)/ 
*Institutional Research/*Minority Role/*Professional Training/*Program Description/ 
*Program Devel opment/*Program Improvement/*Program Proposal /*Research and Developmertt 
Centers/*Research Skills/ 



The training institute will be conducted on the campus of The Ohio State 
University. The general purpose of the institute will be to design a model 
program to be utilized by existing R&D facilities of larger institutions in 
the preparation of instructional programs and procedures to be implemented 
by predominantly Black and other small institutions. The pilot programs pro- 
posed for the praticipating institutions will be designed to meet the following 
objectives within the said institutions: 

1, To assist Black colleges and universities of strong potential to develop 
centers for educational research and development, 

2. To train professors to implement research programs using existing materials, 
and to further develop and/or modify existing program and material packages, 

3. To identify a sequence of experiences adequate to train researchers and 
other personnel at the undergraduate and Masters' level, 

4, To facilitate the functions of the Office of Institutional Research, 

Upon completion of the Academic Year Institute, the R&D team will return to their 
respective campuses and implement a program designed to meet the R&D needs of 
the said institutions. 



The Ohio State University 

College of Education, Faculty of Educational Development 
29 W, Woodruff Avenue 
Columbus, Ohio 43210 

(fi14) 4??-4a72 . 
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Strengthening Institutional Research and Development Capability in Institutions 

with Substantial Enrollment of Chicano and Native American Students 
Hale, James A. 

Albuquerque, New Mexico 87106 i 
University of New Mexico, College of Education, Dept of Ed Administration . i 

February.! 1972 , ^ _ , . ^ ! 

*Change^Agents/*Educational Development/*Educational Research/*Insti tutional Research/ 
♦Management Educatior\/*Minority Role/*Professional Training/*Program Description/ ' 
♦Research Skills/ I 



A consortium of developing two-year and four-year colleges will provide two 
or more staff members one calendar year's training at The University of New 
Mexico. Participants will be Native American or Chicano staff members from 
developing institutions serving substantial members of Native American and 
Chicano students. 

It is the purpose of the training program to provide research and develop- 
ment skills and management skills appropriate to institutional change-agents 
roles in the target institutions of higher education. It is a further purpose 
of the program to develop an exportable training model to institutions of 
higher education who might assist other consortia of ethnically stratified 
developing institutions. 



University of New Mexico 
College of Education 

Department of Educational Administration 

Albuquerque, New Mexico 87106 

(505) 277 ^533 — 
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A Training Model for Change Agents in State Education Agencies ^ 

Havelock, Ronald 6. ' 

Ann Arbor, Michigan i 

The University of Michigan i 

.lunfi, 1970, OE C-0 -8-080603-4535 (OlOK 45 P. ' 

^Annotated Bibliographies/*Curriculum Guides/*Change Agents/*On The Job Training/ ^ 

^Institutes (Training Programs/*Educational Innovation/*Administrator Education/ ' 

♦Simulation/ I 

I 



This Interim report outlines a plan to train state education department 

personnel as change agents. . 

Attitudes, values, knowledge, and skills that trainees must have are listed. 

Commitments that state education departments must make are specified. Twenty ! 

changes in the attitudes, values, knowledge, and skills of trainees that ' 
training should produce are described. Thus the role of change agent is defined, ( 

The plan calls for three workshops totalling ten days during a year of I 

assigned reading and on-the-job role practice. There are outlines of the t 

first eight workshop units giving topics for lecture and discussion, assignments , 

for reading and writing, skills for demonstration, and situations for role- , 
playing. A case simulation exercise is suggested. 

Means of reinforcing workshop learnings on the job include the requirement I 

that each state send two trainees and that each trainee keep a log of readings j 

and activities. ; 

A reading list of eighteen items, each annotated and rated "essential" 

or "desirable," is included. Four pages by Floyd C. Mann, "Handling Mis- ' 

understandings and Conflict," are inserted. ' 

Some desiderata for integrating the new role of change agent into state | 

departments are stated. ' 

The program's goal is not trainees but beneficial change in American 

education. Thus, through criteria and means for evaluating the training's i 

effects and the trainees' activities are suggested, the question of what inno- ^ 
vations may prove the program useful can only be posed. 

The companion textbook is A Guide to Innovation in Education. (AS) f 



The University of Michigan 
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Internal Training of Title III Specialists: An Imperative for '. ^ 

Changing Educational Practice 
Hopkins, John E, ' 
Columbus, Ohio ' 
The Ohio State University, College of Education * 

iqfi7, Vol. fi. 13A>lAn p . I 

*Manpower Development/*Educational Change/*Research and Development Centers/ 
*Specialists/*Training/*Teacher Programs/*Manpower Needs/*Educational ' 
Development/*Educational Improvement/ ' 

I 

. . L 

A lack of trained development and demonstration specialists slows down i 

the possibilities for change in American schools. A proposal is outlined j 

which would develop training programs for such specialists at educational . 

centers authorized by Title III of the Elementary and Secondary Education j 
Act of 1965. These centers work from within school districts for educational 

change. Internal training programs of the proposal would provide needed ! 

trained specialists from the ranks of classroom teachers. The proposal ' 

outlines the following: underlying assumptions; trainee qualifications; I 
selection of trainees; solution of district problems by training; content . \ 

of the initial training program; obtaining objective reports on the i 
program; organization of the program; future advantages of such a proposal. 

The role of other ESEA centers and personnel needs in educational research ' 

and development are also discussed, (GS) I 
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Urban Educational Researchers Training Institute 

I i 

McAdoo, Hariette P. ' 
6th Street & Howard Place, N.W. , Washington, D.C. 20001 i ' 

Howard University, School of Social Work, Research Sequence ' 

npcpmhpr. 1971 ' 

*Data Collection/*Data Analysis/*Educational Developnient/*Educational Research/ | 
*Evaluation/*Field Experience/*Institutes (Training Programs )/*Minority Role/*Profess1onal 
Training/*Program Description/*Research Design/*Research Projects/ I 

I 

= ^ " ^ I 

The primary focus of the Institute will be to develop an inservice model for i 

training personnel who are actively involved in designing, evaluating, and I 
directing research and developmental programs in educational systems and agencies 

within the urban community. Trainees will be chosen to represent the diverse j 

ethnic and cultural groups in our society. While learning the fundamentals of ' \ 

research planning, data collection, data analysis, and report writing appropriate I 

to the school system, trainees will be expected to view the research process I 

within the socio-cul tural environment within which they work. Howard University's | 

unique position as a Black institution within a predominately Black urban community j 

and the pool of young academic researchers at Howard will be extensively involved | 
within the training program. 

The Institute will be designed around the natural school year. Trainees will ' 

spend two weeks on campus, prior to the opening of school in an intensive study | 

of research design, data collection, and statistical analysis. They will evolve | 
the research design appropriate for their own programs. The actual research will 

be conducted during the school year, with two weekend sessions scheduled during ' 

the year for additional training and on-going evaluation. Continuous support . ' 

will be given the trainees through the use of the weekend sessions, monthly | 

contacts, and phone calls. On-site visits will be made by the professional staff ^ 

when needed. At the end of the school year they will return to the campus for ^ 

two weeks of statistical analysis, report writing, and evaluation. j 

Evaluative instruments will be developed for the Institute training procedures 
and for inservice guidance and support during the research process. Such instruments^ 

will be disseminated to other teacher or research training institutions or public ' 

school systans involved in similar inservice research training. ' 

j 
I 
I 
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A Content-Process Model for Training Educational R&D 

Practitioner, Diffuser, and Developer Teams 
Miskel, Cecil G. 
Bailey Hall, Lawrence, Kansas 
University of Kansas, Schoo] of Education 

December, 19 71 

*Diffusion/*Educational Change/*Educational Development/*Educational Technology/ 
♦Field Experience/*Individual Instruction/*Professional Training/*Program Descriptio'n/ 
*Research Skins/*School Personnel/*Summati ve Evaluation/*Team Training/ I 



A content-process model for training three levels of educational Research 
and Development personnel is proposed. 1) Developers to translate the basic 
science into technology; 2) Diffusers to package, distribute, and introduce 
this technology into school practice; 3) School practitioners to test-market 
and improve the R&D training content and process. 

The proposal has two major innovative strategies for training R&D personnel 
to develop, diffuse, and practice tha nev/ R&D skills. (1) A training content 
model designed to collect the new R&D skills, integrate them into a PPBS frame- 
work, and provide climate indicators and social indicators for facilitating 
change. (2) A training process model designed to produce the practitioners, 
diffusers, and developers as a coordinated unit. 

Monitored practice, through one school year, on the unique problems of the 
participating districts insures practical training content and process. 
Monitored practice also yields monthly in-line growth charts of the frequency 
of using the recently learned R&D skills. These charts will also permit early 
corrective action and valid program evaluation. Custom tailoring the training 
to meet the needs of each trainee and each school district further guarantees 
practicality. Thus all three levels of R&D trainees will learn by doing. 



University of Kansas 
School of Education 
Bailey Hall 
Lawrence, Kansas 
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A Design of New Patterns to Train Research, Development, Demon/Dissemination Evaluation 
Personnel in Education, Vol. I: Design of the Training Program 

800 Brazos Street, Austin, Texas 78701 
Southwest Educational Development Laboratory 

December 18. 1970 (Submitted to NCERD) 89p. ^ . 

*Educational Programs/*Evaluation/*Personnel Needs/*Diffusion/*Eclucational 
Research-Educational Reseachers/*Research Design/*Educational Development/ 
*Training Objectives/ 

This training program was designed to train educational research and research- 
related personnel in four areas: research, development^ diffusion, and evaluation, 
The seven primary skill areas relating to the functions of RDD&E are: 1) con- 
ceptualizing issues and orocesses in education-, 2) designing techniques to carry 
out educational goals; 3) setting educational objectives; 4) measuring and 
evaluating educational objectives; 5) summarizing and communicating outcomes; 
6) implementing outcomes; 7) Identifying and incorporating attitudes, values, 
and practices of minority groups in the educational process. Existing manpower 
needs in RDD&E were assessed through indepth interviews at 21 representative 
national agencies or Institutions which engaged in one or more phases of RDD&E, 
and through an Intensive review of professional literature. The above-mentioned 
seven skill areas emerged as the focal points of the proposed training program. 
After reports by Task Forces and consortia on strategies, a systems approach 
utilizing individualized instruction was chosen. The proposed training system 
had four subsystems: diagnosis, training, placement, and management. Two 
types of training were envisioned: 1) modularized packages of material stressing 
conceptualization of specified skills to be used by an individual in his normal 
work or education environment; 2) a structured internship experience following 
development of a critical mass of skills and based on the individual's 
previous experience and perceived future professional role. The Developmental 
Process System was in developing this training program, progressing through six 
stages: 1) context analysis; 2) conceptual design; 3) product design; 4) pilot 
test; 5) field test; 6) marketing and diffusion. The consortium felt a four 
year period was necessary to fully test the full training program as designed. 
(MB) 
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A Training Program for Urban School 


Research and Evaluation ^ 


Personnel 


! 


Wise, Arthur E. 


1 


5801 South Ellis Ave,, Chicago, 111 


. 60637 I 


The University of Chicago, Graduate 


School of Education i 


February • 1972 


1 



"^Educational Change/*Educational Environment/*Educational Problenis/*Educattonal Researjch/ 
^Educational Sociology/*Evaluation/*Insti tutional Environn]ent/*Minority Role/*Professi|Ona 
Training/*Program Descrf ption/*Program Development/*Progra]n Evaluatlon/^Research Skills/ 

*Urban Education/ ' 

1— 

The Graduate School of Education of the University of Chicago proposes to i 

engage in developing a model for the preparation of educational professionals I 
who can evaluate and assess programs in urban schools. Currently, traditional 

educational research techniques are rejected by Black clients; a new orientation | 

to research in urban schools must be assumed. The perspective proposed is one ^ 

that permits a realistic examination of individuals in institutional settings. ! 

The researcher must learn to view the problem from the perspective of the I 

social actor in the institution. The Objectives of the training program are: I 

(1) to train personnel for a new role in an effort to improve the investigation , 

of problems in urban education and to evaluate on-going programs; (2) to develop j 
a new research perspective for a more meaningful analysis of the environment of 

inner city schools; and (3) to design and operational ize a model for training I 

urban school researchers which can be replicated. ( 

Our goal will be to encourage ten students to engage in urban school research j 
and evaluation projects. In so doing, we will be exposing them at a critical 

stage in their career development to urban problems. As a result of this ex- ' 

perience, many will be induced into research and evaluation careers in urban ' 

school systems and in other relevant agencies. Those who choose academic J 

careers may have had their research interests channeled by this experience. ^ 

We propose to hold an oper competition among our students for proposals. 

The project will be made a highly visible one and, it is anticipated, will ^ 
result in the selection of projects developed by our most outstanding students. 



The University of Chicago 
Graduate School of Education 
5801 South Ellis Avenue 
Chicago, Illinois 60637 
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The Need for Research, Development, Dissemination, and Evaluation Personnel in ^ 
Education. Design Document I for the Midwest Educational Training Center. ■ 
Evans, Geraldine . » 

Minneapolis, Minnesota i 
Upper Midwest Regional Educational Laboratory, Inc. i 

December 14, 1970, 42 p., 0EC-3-7-n62870-3q69 [ 

*Administrative Personnel, Data Co71ection/*Educatiohal Programs, Evaluation, ^ 
Industrial Personnel, Manpower Needs/*Personnel Needs/*Project Applications/ ' 
*Regional Laboratories, Research and Development Centers, Research Needs, Research 1 
Skills/*Staff Utilization/ i 

^ ^ ^ ^ L 

This document, one part of a project to train personnel for educational development | 
and evaluation, features data that represent judgments of informed experts on 
personnel needs in various activities in four kinds of research and development 

agencies: (1) local, intermediate, and State educational agencies, (2) regional I 

laboratories and research and development centers, (3) colleges and universities, ' 

and (4) educational or training divisions of industrial firms. A summary of ! 

the data reveals that evaluation skills are in greatest demand relative to supply 1 

and that a great need exists for professional supervisory personnel who could I 

organize and supervise development and evaluation activities. Related documents | 

are EA 003 900, EA 003 901, and EA 003 902. , 
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Field Reader Catalog 

Educational Researchers 



May. .1970 

♦Catalogs /*Eva1udtion/*Specialists/*Project Applications/*Educationa1 Research/ 
♦Educational Researchers/ 



The Field Reader Catalog is a compilation of subject area specialists under 
contract to the National Center for Educational Research and Development (NCERD) 
of the Office of Education. These specialists provide technical reviews and 
evaluations of proposals submitted to NCERD. Each field reader supplies the 
information used about him in the catalog. The Field Reader Catalog consists 
of two major sections: Field Reader Indexes and Field Reader Resumes. Included 
in the Indexes are: 1) Name Index--an alphabetical listing of the name of each 
field reader; 2) Regional Index— a listing of field readers by the Department 
of Health, Education, and Welfare Region in which they reside; 3) Specialty 
Index--an alphabetical listing by subject specialty of field readers by their 
subject areas of expertise. The Resume section is arranged numerically by 
field reader dccession number. A biographical sketch of each field reader is 
included. Access from indexes to resumes is provided through the accession 
numbers assigned to each field reader. (MB) 
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Handbook of Research on Teaching 

Gage, N. L. (Ed.) 

Chicago, Illinois 

Rand McNally & Company 

iqfi:^. 1?1R pp. 



THEORETICAL AND CONCEPTUAL ISSUES 
UNDERLYING RDD&E PROCESS PAPERS 
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Change and Innovation in Elementary and Secondary Organization 

Hillson, Maurie & Hyman, Ronald T. (Eds.) 
New York, New York . 
Holt, Rinehart and Winston, Inc. 

1965. 435 pp. : 
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Perspective on Educational Change 

Miller, Richard I. 
New York, New York 

Appleton-Century-Crofts, Division of Meredith Publishing Company 

1972. 392 pp. 



RESEARCH ON EDUCATIONAL R&D PAPERS 
1.4 



A Review of Research and Development Centers bupported by tne 
of Education 

Athens, Georgia 30601 

Journal of Research and Development in Education, College of Education, University of 

IQfiR, 909 pp . 
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The Measurement of Efficiency of Scientific Research 
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Lipetz, Ben-Ami 
Carlisele, Massachusetts 
Intermedia, Inc. 

iQf^'^ 
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A Review and Critique of Studies of Educational RDDE Training 

Smith, Mary Lee 

University of Colorado 

American Educational Research Association 

September. 1971. QEG>n-71-nfi17 (R?n), ?qn p . 
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A Synthesis of the Results of Research on the Training of Research and Research- 

Related Personnel in Education 
Smith, Mary Lee, Anderson, Ronald D., Gephart, William J, 

American Educational Research Association' 

November, 1971, OEG-0-71-0617-520, 57 p, ' " 



\ FWL 014 0108 

An Exploratory Study of Selected Variables Related to the Training and Careers of 

Educational Research and Research-Related Personnel (Final Report), Appendix 
Worthen, Blaine R,, Byers, Maureen L. 
Washington, D.C. 

American Institutes for Research 

Dec ember. 1970, OEG-0-9-180240-3757 (010), 389 p, 
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Research Training Activities Stimulated by the AERA Task Force on Research 

Training, 1969-70 
Worthen, Blaine R., Popham, W. James 

* 

American Educational Research Association 

July. 1970. OEG-0-9-180240-3757 (010). 14p. 
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A Study of Selected Factors Related to the Training of Researchers, Developers, DiffJserd 

and Evaluators in Education 
Worthen, Blaine R., Anderson, Ronald D., Byers, Maureen L. 
Washington, D.C. 

American Educational Research Association 

Nnv^hPr , 1971. QER-n-7 1 -OfiU (F.?n), .-^41 
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Conmunication and Status: The Dynamics of a Research Center ~ 

Smith, Alfred G. 

University of Oregon, Eugene Oregon 

The Center for the Advanced Study of Educational Administration 

J96f . 58 pp. _^ 
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Discovering Children: A Responsive Program for Pre-School Children 



Hotel Claremont, One Garden Circle, Berkeley, California 94705 
Far West Laboratory for Educational Research and Development 
28 p. 



Far West Laboratory for Educational Research & Development 
Hotel Claremont, One Garden Circle, 
Berkeley, CAlifornia 94705 
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Organization of School Research Departments 

Hood, Paul D* 

1 Garden Circle, Hotel Claremont, Berkeley, California 94705 
Far West Laboratory for Educational Research & Development 



Far West Laboratory for Educational Research & Development 
1 Garden Circle, Hotel Claremont 
Berkeley, California 94705 
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Educational Development Case Study: An Elementary Science Information Unit j 

Hutchins, C. L. * 

1 Garden Circle, Hotel Clarenont, Berkeley, California 94705 ' 

Far West Laboratory for Educational Research and Development ' 

January K 197K 6Rp. ^ | 
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G.O.A.L.S. Goal-Setting for Organizational Accountability: A Leadership 
Strategy, Instructions (1), Glossary (2), and Team Leaders 's Instructions (3) 

Durham, North Carolina 27701 

National Laboratory for Higher Education 

1971. 9 p. . 
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"The Generation of Information to Support Long-Term Manpower Studies of and Planning 
for Training Programs in Educational RDD&E", Vol. II, Parts 1 & 2, Vol. IV. f 
Porter , Bette C . , ed . ' 
Monmouth, Oregon 97361 i 
Teaching Research-Oregon State System of Higher Education i 
January. 1972, QEG-0-70-4977 . ! 



Teaching Research-Oregon State System of Higher Education 
Monmouth, Oregon 97361 
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A Guide to Securing and Installing the Parent/Child Toy-Lending Library ^ 

Rosenau, Fred, Tuck, Betty • 
1 Garden Circle, Hotel Claremont, Berkeley, California 94705 I 
Far West Laboratory for Educational Research and Development i 
April, 1972 • 
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The South Carolina Pilot Program for Information Dissemination, A ' 
Narrative Report, July, 1970 to December, 1971 ^ 
South Carolina Department of Education ' 

. I 

South Carolina Department of Education ' 
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Preparing Research Personnel for Education, a PDK-AERA Study 



Clark, David L,, Worthen, Blaine R., ed, 
Washington, D,C, 

American Educational Research Association 
1967. 92 p> 



American Educational Research Association 
Washington, D- C. 
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Educational Project Management 

Cook, Desmond L. 
Columbus, Ohio 

Charles E. Merrill Publishing Company » A Bell & Howell Company 

1971 > 243 pp. 
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Determining Instructional Purposes, Unit 1: Setting Goals 

Gall, Joyce P,, Rosenoff, Wayne York, Linda Oakley, Gail M, 

1 Garden Circle, Hotel Claremont, Berkeley, California 94705 

Far West Laboratory for Educational Research and Development 

197K 105 p. 



Far West Laboratory for Educational Research and Development 
1 Garden Circle, Hotel Claremont 
Berkeley, California 94705 
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Determining Instructional [Purposes, Unit Z: Analyzing Problems r 

Gall, Joyce P., York, Linda J., Kertzberg, Richard J. ^ 
1 Garden Circle, Hotel Claremont, Berkeley, California 94705 i 
Far West Laboratory for Educational Research and Development i 
1971 , 139 p. [ 
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Far West Laboratory for Educational Research and Development 
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Berkeley, California 94705 
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Decermining Instructional Purposes, Unit 3: Deriving Objectives 

Jenks, Charles L., Bostick, Nancy G., Otto, J. Gregory 
1 Garden Circle, Hotel Claremont, Berkeley, California 94705 
Far West Laboratory for Educational Research and Development 
1971 



Far West Laboratory For Educational Research and Development ' 
1 Garden Circle, Hotel Claremont ' 
Berkeley, California 94705 ' 
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Principles and Practice of Instructional Technology, Workbook 



424 University Avenue, Palo Alto, California 94302 

General Programmed Teaching, A Division of Commerce Clearing House, Inc. 

158 p. 




General Programmed Teaching, A Division of Commerce Clearing House, Inc. 
424 University Avenue 
Palo Alto, California 94302 
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Research in Elementary School Curriculum 



Johnston, A. Montgomery, Burns, Paul, C, ed. ' 

470 Atlantic Avenue, Boston, Mass. 02210 i 

Allyn and Bacon, inc. i 

1970. 543 p. 
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Allyn and Bacon, Inc. 

470 Atlantic Avenue 

Boston, Massachusettes 02210 
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Developing Attitude Toward Learning 
Mager, Robert F. 

2165 Park Boulevard, Palo Alto, California 94306 
Fearon Publishers 

1968, 104 p. 



Fearon Publishers 

2165 Park Boulevard 

Palo Alto, California 94306 
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Rational Planning in Curriculum and Instruction; Eight Essays 



1201 16th Street, N.W., Washington, D.C. 20036 

National Education Association 

1967. 203 p. 



National Education Association ' 
1201 16th Street, N. W. ' 
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Instructional Improvement Training Series, Junior and Community College 
Division, Accountable Learning Systems Program 

Durham, North Carolina 

National Laboratory for Higher Education 

im L_ 



National Laboratory for Higher Education 
Durham, North Carolina 
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Developing Executive Leaders, (Book Review) ~" 

Bursk, Edward C, Blodgett, Timothy B. (Ed.) 
Cambridge, Mass. 
Harvard University Press 

197K Personnel Journal, July, -1972, Vol. 51 > Number 7 (Reference), 191 p. 



Harvard University Press 
Cambridge, Mass. 
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Drugs, A Trip to Nowhere, (Book Review) 



Personnel Journal. July, 1972. Vol. 5K Number 7, 470-472 p. 
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Interviewing Techniques for Non-Personnel txecutives . 



Personnel Journal 

July. 1972, Vol. 51, Number 7. 474 p. 
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The Dartnell Corporation 

Personnel Journal, July, 1972, Vol. 51, Number 7 
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The Bookshelf, (Book Reviews) ] 



Personnel Journal. July. 1972, Vol. 51. Number 7. 533 p. 



i 



ERIC 



FWL 024 0232 i 



Paper Blizzard: A Communication Film Package to Aid Information Users & Library Sciehce 

Students (From the 33rd Annual Mtg. of the American Society for Information Sciencfe) 
Tressel , George' ' 
Columbus, Ohio i 
Battel e Memorial Institute i 
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Project Abstracts of the Research Training Branch National Center for ' 
Educational Research and Development (12 cooies) ' 

1 Garden Circle, Hotel Claremont, Berkeley, California 94705 i 
Far West Laboratory for Educational Research and Development i 
March, 1972, 19 p. ^ I 
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Far West Laboratory for Educational Research and Development 
1 Garden Circle, Hotel Claremont 
Berkeley, California 94705 
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Keport on the Main Field Test of the Alert Information System ^^^^ 

1 Garden Circle, Hotel Claremont, Berkeley, California 94705 
Far West Laboratory for Educational Research and Development 
November, 1971 , 119 p. 



Far West Laboratory for Educational Research and Development 
1 Garden Circle, Hotel Claremont 
Berkeley, California 94705 
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Design Document III for METC: The Management System 

Ammentorp, W. , Raymond, R., Hendrix, V. 

1640 East 78th Street, Minneapolis, Minnesota 55423 

Upper Midwest Regional Educational Laboratory 

December 14. 1970. 93 p. 
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The Proposal for the Midwest Educational Training Center, Training ^ 
for Development and Evaluation in Education ' 
Ammentorp, W., Welch, W., Evans., G. ■ 
1640 E. 78th Street, Minneapolis, Minnesota 55423 i 
Upper Midwest Regional Educational Laboratory i 
December 14, 1970, 114 p. - 
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A Training Program for Developers & Evaluators of Educational ^ 

Products & Supplement to Proposal } 

Baker, Eva L. ' 

Los Angeles, California 90024 ^ ' i 

University of California, Los Angeles i 

Oficember. 1970. QEC--Q-70-4765 (520), Office of Educa tion, 89 p. « 
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Development of Trairr'ng Resources for Educational Extension 

Services Personnel 
Banathy, Bel a H. 

1 Garden Circle, Hotel Claremont, Berkeley, California 94705 
Far West Laboratory for Educational Research and Development 
March, 1972 ( Date Transmitted), 75 p. 
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Investigation of Factors Influencing the Training of Educational ' 
Researchers, Final Report i 
Bargar, R. R., Okorodudu, C. P., Dworkin, E. P. . ' 
1314 Kinnear Road, Columbus, Ohio 43210 i 
The Ohio State University Research Foundation i 
May. 1970. OE-6-10-146, 119 D. ' 



The Ohio State University Research Foundation ' 
1314 Kinnear Road ' 
Columbis, Ohio 43210 | 
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1971 AERA Research Training Sessions f 

Byers, Joe L. ' 

1126 16th St., N.W. Washington, D.C. 20036 i 

American Educational Research Association i 

May, 1972. OEG-0-71-1173. 117 p. ' 
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American Educational Research Association 
1126 16th Street, N. W. 
Washington, D. C. 20036 
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Technical Paper No. 2: Description and Rationale of the Training 
Consortium 

University of Colorado, Boulder, Colorado 80302 

Colorado Center for Training in Educational Eval . & Development 

0EC-0>70-4770 (520). U. S. Office of Education, 30 p. 



Colorado Center for Training in Educational Evaluation and Development 
University of Colorado 
Boulder, Colorado 80302 
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Technical Paper No. 3: A Proposed Design for Training Educational 
Evaluators and Develope, Appendices A-D 

University of Colorado, Boulder, Colorado 80302 

Colorado Center for Training in Educational Eval . & Development 

OEC-0-70-4770 (520). U. Office of Education, 101 p. 
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Design Document I for METC: JUe Need for Research, Development, 
• Dissemination & Evaluation Personnel in Education 
Evans, Geraldine 

1640 East 78th Street, Minneapolis, Minnesota 55423 
Upper Midwest Regional Educational Laboratory 

December 14. 1970 > 39 p, 
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Educational Management, Basic Program Plan ^ 

I 

1 Garden Circle, Hotel Claremont, Berkeley, California 94705 i 
Far West Laboratory for Educational Research and Development i 

March, 1972 [ ! 
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A Role for State Education Agencies in the Model Cities Program: 

The New Jersey Experience, Report 
Fitzgibbon, J. Gerald 

225 West State Street, Trenton, New Jersey ns6" 

New Jersey State Dept. of Education, Office of Model tnies 

April , 1971 . 31 p. 



New Jersey State Dept. of Education ' 
Office of Model Cities ' 
225 West State Street ' 
Trenton, New Jersey 08625 
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Prop, for the Design of a New Pattern for Training Res/Oev/Demonstration/Disseminatibn, 

& Eval I Personnel in Ed., Activities on the Design Phase, Vol. I of IV, Appendices' 
Fletcher, Jerry L. ' 
Monmouth, Oregon 97361 ' 
Oregon State System of Higher Ed., Teaching Research Division < 
nprpmhftr Ifi, 1Q7n. nFG-0>7Q-4977 (Nat'l Cente r for Ed. Res, & Dev.). 149 p. | 
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Oregon State System of Higher Education 

Teaching Research Division ^ 
Monmouth, Oregon 97361 

L 



ERLC 



FWL 027 0254 



A Proposal for the Development of Course Content and Instructional Materials/Aids for 
the Training of Educational Research, Development, Diffusion, & Evaluation Personneil 
Cropper, George L. 

135 N. Bellefield Ave., Pittsburgh, Pennsylvania 15123 
American Institutes for Research 

May 28. 1970, 51 P. 



American Institutes for Research 
135 N. Bellefield Ave. 
Pittsburgh, Pennsylvania 15123 
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Program Design for Syracuse Center for Evaluation and Research 

Training (SCERT) 
Harootunian, Ber j , Samph, Thomas 
400-406 Huntington Hall, Syracuse University 
SCERT 

December. 1970. OEC-0-70-4767 Office of Education, Bureau of Research, 129 p. 
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R&D Personnel in 
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Learning Research and Development Center 
University of Pittsburgh 
Pittsburgh, Pennsylviania 15123 



Final Report on Project to Design New Patterns for Training 
Education, Part I: Goals & Activities of the Design Proj 

University of Pittsburgh, Pittsburgh, Pennsylvania 15123 ' 
Learning Research and Development Center ' 
nprpmhPr IR, IQTO, nrr^lQ-.70-4771 (52QK 21 p. ' 
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Final Report on Project to Design New Patterns for Training R&D Personnel in Educat'ion, 
Part I IB; Rationale & Description of the Training Consortium 



University of Pittsburgh, Pittsburgh, Pennsylvania 15123 

Learning Research and Development Center 

nprpmhpr 1«, 1Q7n, nrr-10-70-4771 (520K 21 p . 



Learning Research and Development Center J 
Univejrsity of Pittsburgh i 
Pittsburgh, Pennsylvania 15123 , 
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Final Report on Project to Design New Patterns tor Training K & u r 
Personnel in Education, Part IIA: Designs of the Training Prog, i 

University of Pittsburgh, Pittsburgh, Pennsylvania 15123 \ 
Learning Research and Development Center i 

Dec ember 18 > 1970, OEC-10-70-4771 (520), 51 p. ' 
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Learning Research and Development Center ' 
University of Pittsburgh ' 
Pittsburgh, Pennsylvania 15123 ' 
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Final Report on Project to Design New Patterns for Training R&D 
Personnel in Education, Part IIC: Scope of Work, Appendices A, B, & C 

University of Pittsburgh, Pittsburgh, Pennsylvania 15123 
Learning Research and Development Center 

December 18, 1970, OEC-10-70-4771 (520), 7 p. 



Learning Research and Development Center ' 
University of Pittsburgh * 
Pittsburgh, Pennsylvania 15123 | 
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Applied Organization Specialists for Public Ed: A Prop tc Design, Organize, Operate & 
Evaluate a Multi-Institutional Consortium-Sponsored Research & Devel . Training Center 

Suite 303, 3311 W. Manchester Blvd., Inglewood, Calif. 90305 
PEOR Urban Associates 

71 n. 



PEDR Urban Associates 
Suite 303 

3311 W. Manchester Blvd. 
Inglewood, Calif. 90305 
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A Training Program for Developers & Evaluators of Educational 

Products, Final Report, App for Res Support & Briefing Sunwary 
Popham, W. James 
Los Angeles, California 90024 
University of California, Los Angeles 

Decpmber. 1970. QEC-Q-7Q-4765 (520). Office of Educal ion_^_9_p . 



University of California 

Los Angeles, California 90024 



ERIC 



FHL 027 0270 



A Design of New Patterns to Train Research^ Development, DCTOn/Dissemindtion Evaluation 
Personnel in Education, VoK II: Scope and Development Process of the Training Prog. 



800 Brazos Street. Austin, Texas 78701 
Southwest Educational Development Laboratory 
pprAmhPr 18. 1Q70 (Sub mitted to NCERD), 278 p. 




Southwest Educational Oevelopiiient Laboratory 
800 Brazos Street 
Austin. Texas 78701 
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Prop, to Design New Patterns for Training Res., Dev., Detfonstration/ 

Dissemination, S Eval. Personnel in Ed., Vol. Ill of III, Apend. 
Stufflebeam, Daniel L. 
1314 Kinnear Road, Columbus, Ohio 43212 
The Ohio State University Research Foundation 

December 18. 1970. OEC-0 -70-4766 (520) . Office of Educa t i on , 533j)^ 



The Ohio State University Research Foundation | 
1314 Kinnear Road 
Columbus, Ohio 43212 
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Technical Progress Report: Program to Onorational i zc a New Training 

'^attern for Training Evaluation Perscnnr?! in Education 
St.jfflebeam, D. L., Root, 0. K. 
1314 Kinnear Road, Colunbus, Ohio 43?12 
The Ohio State University Research Foundation 

January 31. 19 72. OEG-0-71 -105 1 (Office of Education) , 68 



The Ohio State University Research Foundation 
1314 Kinnear Road 
Columbus. Ohio 43212 
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New Patterns for Training R, D, D, and E Personnel, Final Report, 

Turner, Richard 

Room 227, Education, Bloomington, Indiana 
Indiana University Foundation 

December 15, 1970 



Indiana University Foundation 

Room 227 

Education 

Bloomington, Indiana 
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Proposal: National Center tor tne iraimng of EuUcatlonal 
Resource Agents to Serve Rural Minorities: Project Mgmt. 
Tuskegee Institute, Alabama 



December 15, 1970, 241 p. 
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Nat*-! Center for the Training of Ed Resource Agents to Serve Rural Minorities/The Prep^ara 



tion of Problem Solving/Devel/Diff Personnel to Serve Rural/Minority/Culturally Ltd 
Tuskegee Institute, Alabama 

National Federation for the Improvement of Rural Education 
nprpmhpr 15> 1970> 241 p. 
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National Federation for the Improvement of Rural Education 

Tuskegee Institute 

Alabama 
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Design Document II for METC: Conceptual Papers Defining TRe ^ 

Knowledge & Skills Required to Function as Ed. Dev. & Eval . ' 

Welch, W. , Hendrix, V., Johnson, P., Terwilliger, J. ' 

1640 East 78th Street, Minneapolis, Minnesota 55423 i 

Upper Midwest Regional Educational Laboratory i 

Decembe r 14, 1970, 141 p. « 
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Upper Midwest Regional Educational Laboratory 
1640 East 78th Street 
Minneapolis, Minnesota 55423 
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ECS Assessment Workshop for State Department Personnel , Final 

Report, Part I 
Womer, Frank B., Lehmann, Irvin J. 
Denver, Colorado 

Education Commission of the States 

September 30, 1971, OEG-0-71-4554 , Office of Education, NCERD, 51 p. 
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Education Commission of the States 
Denver, Colorado 
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The Status of Vocational Education, R, D, D, and E Personnel 

Schroedeft Paul E. 
Minneapolis, Minnesota 55455 
University of Minnesota 

Final Report, Project Nn. 1-»13?, firant Nn. nPfi-O-yi-A'^ fin , Marrh^ 107^ 
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EDUCATIONAL RDO&E PERSONNEL AND TRAINING INDEX 

PART 4 



Note: This index identifies, by the Far West Laboratory accession 
number, each descriptor that has been employed. These 
accession numbers will be found in the top right-hand section 
of each form. 
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ABE: 021 0132. 

ABE Program Development: 021 0132. 
Abstracting: 021 0160= 022 0186. 
Abstraction Levels: 021 0118. 
Academic Achievement: 021 0140, 021 0145. 
Academic Education: 012 0063. 
Academic Performance: 021 0145. 
Academic Pxe cords: 021 0148. 
Accreditation (Institutions): 012 0020. 
Achievement Gains: 022 0197, 022 0222. 
Achievement Rating: 021 0145. 
Achievement Tests: 012 0028. 
Action Programs: 012 0035. 
Action Research: 021 0140. 
Activities: 012 0020. 
Activity Units: 021 0125. 
Administration: 012 0048,014 0105. 
Administrative Change: 021 0143. 
Administrative Personnel: 036 0279. 
Administrative Policy: 012 0004. 

Administrator Education: 022 0207, 027 0242, 027 0258. 
Administrator Guides: 022 0220. 
Administrator Role: 012 0051, 027 0242. 
Admission Criteria: 014 0101. 

Adoption (Ideas): 012 0049, 012 0056, 012 0061, 012 0062, 012 0073. 

Adult Education: 012 0036. 

Adult Educators: 012 0057. 

Advisory Committees: 012 0023. 

Aerospace Industry: 014 0100, 014 0102. 

Affective Objectives: 022 0195, 022 0215. 

Age Differences: 012 0062. 

Agriculture: 012 0060. 

Agronomy: 012 0038. 

Alexandria: 012 0059. 

Analytical Criticism: 012 0086, 022 0205. 

Annotated Bibliographies: 022 0167, 022 0176, 022 0178, 022 0223, 

027 0256. 
Applied Linguistics: 021 0115, 
Arithmetic: 021 0121. 
Astronomy: 012 0027. 
Audiences: 012 0049, 012 0057, 021 0138. 
Audioinstructional Aids: 012 0122. 
Audiovisual Aids: 024 0227. 
. Authors: 012 0028. 
Auto- Instructional Aids: 022 0217. 
Auto-Instructional Methods: 022 0175, 022 0217. 
Auto-Instructional Programs: 022 0217. 
Bayesian Statistics: 022 0204. 
Beginning Reading: 021 0146. 
Behavior: 012 0023. 
Behavior Change: 022 0200. 
Behavior Patterns: 022 0200. 
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Behavioral Objectives: 021 0113, 021 0118, 021 0124. 021 0127, 021 0128, 

021 0131, 021 0141, 021 0151, 021 0152, 021 0155, 022 0187, 022 0202, 

022 0218. 

Behavioral Science Research: 012 0013, 012 0071, 022 0194. 
Behavioral Sciences: 012 0007. 014 0103, 022 0199. 
Beliefs: 012 0057. 

Berkeley, Ca. Center for the Study of Higher Education: 014 0096. 

Bibliographic Citations: 022 0176. 

Bibliographies: 014 0102, 022 0176, 022 0183. 

Biological Sciences: 012 0061. 

Branching: 022 0175, 022 0212, 022 0217. 

Business: 012 0040. 

Business Administration: 012 0073. 

Career Change: 014 0100. 

Career Choice: 012 0010, 014 0101, 014 0102. 014 0105. 
Career Opportunities: 014 0105. 
Careers: 014 0105. 
Case Records: 021 0144. 

Case Studies (Education): Oil 0001, Oil 0003, 012 0025, 012 0057, 

021 0140, 021 0141, 021 0142, 021 0145, 021 0146, 021 0149, 022 0190, 

022 0205. 
Catalogs: 041 0281. 

Change Agents: 012 0049, 012 0053, 012 0055, 012 0057, 012 0059, 012 0060, 

012 0088, 0?2 0170, 022 0190, 022 0191, 022 0192, 027 0256, 027 0258. 
Chief Administrators: 012 0051. 
Child Development: 021 0117. 
Children: 012 0019, 012 0026. 
Class Management: 021 0131. 
Classification: 012 0009, 021 0152. 
Classroom Research: 012 0010, 012 0067, 022 0197. 
Clearinghouses: 021 0160. 
Cognitive Measurement: 012 0026. 

Cognitive Objectives: 022 0195. 022 0197, 022 0202, 022 0215, 022 0218. 

Cognitive Processes: 012 0039, 021 0129. 

College Administration: 014 0105. 

College Faculty: 014 0104. 

College Mathematics: 022 0193. 

College Role: 012 0066. 

College School Cooperation: 012 0056. 

College Teachers: 012 0058. 

Columbus, Ohio; 014 0093. 

Communication Skills: 022 0169. 

Communication (Thought Transfer): 012 0048, 012 0054, 012 0055, 012 0057, 

012 0060, 012 0063. 
Communications Problems: 022 0170. 
Community Colleges: 022 0210. 
Community Cooperation: 012 0010, 021 0126. 
Community Influence: 021 0112. 

Community Involvement: 021 0120, 021 0142, 021 0159, 022 0207. 
\ Community Programs: 012 0023. 
Community Relations: 021 0161. 
Community Surveys: 012 0023. 
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Comparative Analysis: 012 0015, 012 0039. 

Computer Assisted Instruction: 021 0121, 021 0122, 021 0156, 022 0172. 
Computer Oriented Programs: 021 0113, 021 0119, 021 0148, 021 0153, 

021 0157, 021 0160. 
Computer Programs: Oil 0001. 
Computer Science: 021 0141. 
Concept Teaching: 021 0154. 

Conceptual Schemes: Oil 0003, 012 0055, 012 0089, 022 0195. 

Conferences: 012 0055, 012 0076, 014 0096. 

Conformity: 012 0073. 

Consolidated Schools: 021 0140, 021 0144. 

Constructed Response: 022 0175, 022 0212, 022 0217. 

Construction (Process): 012 0028, 012 0043. 

Consultants: 014 0096, 014 0116. 

Consultation Problems: 012 0059. 

Consumer Science: 012 0084. 

Control Groups: 012 0017, 012 0029, 012 0033. 

Conventional Instruction: 012 0034. 

Cooperative Planning: 012 0067, 022 0167. 

Correlation: 012 0039, 022 0199. 

Cost Effectiveness: 012 0022, 024 0227. 

Costs.: 012 0017, 012 0022. 

Course Content: 012 0034. 

Course Evaluation: 012 0036. 

Course Objectives: 012 0036, 021 0141, 022 0202. 

Course Organization: 022 0208. 

Creative Thinking: 012 0073. 

Creativity: 012 0073. 

Criteria: 022 02?4, 025 0233. 

Criterion Referenced Tests: 022 0179, 022 0216. 

Critical Path Method: 012 0082, 012 0083. 

Critical Reading: 022 0205. 

Cross-Sectional Studies: 021 0145. 

Culturally Disadvantaged: 012 0019, 012 0029, 012 0080, 021 0149. 

Curiosity: 012 0007. 

Curriculum: 012 0030, 012 0052. 

Curriculum Design: 012 0045, 022 0219, 025 0233. 

Curriculum Development: 012 0030, 012 0031, 012 0045, 012 0083, 021 0111 , 
021 0112, 021 0113, 021 0114, 021 0115, 021 0116, 021 0117, 021 0118, 
021 0123, 021 0124, 021 0126, 021 0128, 021 0129, 021 0130, 021 0131, 

021 0141 , 021 0153, 021 0154, 021 0155, 021 0156, 021 0161 , 022 0167, 

022 0169, 025 0233, 027 0242. 

Curriculum Evaluation: 012 0011, 012 0015, 012 0017, 012 0018, 012 0020, 

012 0022, 012 0027, 012 0028, 012 0030, 012 0031, 012 0045. 022 0219. 
Curriculum Guides: 014 0102, 021 0118, 027 0256. 
Curriculum Research: 014 0104, 021 0154, 021 0156, 022 0167. 
Curriculum Study Center: 021 0159. 
Data: 012 0015. 

Data Analysis: Oil 0001, 012 0009, 012 0044, 014 0103, 022 0168, 022 0203, 

022 0213, 022 0216, 027 0245, 027 0264. 
Data Bases: Oil 0003, 012 0070. 



Data Collection: Oil 0001 , Oil 0003, 012 00^, 012 0032, 012 0033, 
022 0169, 022 0189, 022 0203, 022 0213, 027 0264, 036 0279. 

Data Processing: Oil 0001, 012 0032, 021 01A8, 021 0150. 

Decision Making: 012 0004, 012 0013, 012 0016, 012 0021, 012 0023, 
012 0024, 012 0025, 012 0032, 012 0045, 012 0059, 012 0070, 012 0080, 
012 0081 , 022 0172. 

Decision Making Skills: 027 0241. 

Definitions: 012 0004, 012 0007, 012 0009, 012 0011, 012 0013, 012 0086. 
Democratic Values: 012 0066. 
Demonstration Programs: 012 0085. 
Demonstration Projects: 012 0076. 

Demonstrations (Educational): 012 0054, 012 0055, 012 0076, 012 0079. 
Descriptive Writing: 012 0018, 022 0186. 
Design: 012 0009, 012 0069, 022 0208. 
Design Needs: 012 0069. 
Developing Nations: 012 0057. 

Development: 012 0013, 012 0039, 012 0042, 012 0043, 012 0050, 012 0065, 

012 0071, 012 0088, 025 0233. 
Developmental Programs: 014 0094. 
Developmental Reading: 021 0146. 
Developmental Tasks: 022 0188, 023 0225. 
Diagnostic Tests: 021 0117. 

Diffusion: Oil 0001, Oil 0003, 012 0053, 012 0056, 012 0057, 012 0060, 
012 0061, 012 0072, 012 0075, 012 0076, 012 0084, 012 0085, 012 0088, 
012 0089, 014 0095, 021 0113, 021 0116, 021 0118, 021 0127, 021 0138, 

021 0142, 021 0158, 021 0159, 021 0161, 021 0162, 022 0169, 022 0188, 

022 0192, 023 0225, 025 0233, 027 0265, 027 0269. 
Disadvantaged Environment: 021 0149. 

Disadvantaged Youth: 021 0112, 021 0120, 021 0121, 021 0124, 021 0142, 

021 0149, 021 0159, 022 0222. 
Discipline: 012 0073. 
Discovery Learning: 021 0112, 021 0114. 
Discriminant Analysis: 022 0199. 
Discussion (Teaching Technique): 021 0129. 
Distinctive Features: 012 0007, 012 0013, 012 0042. 
Doctoral Degrees: 014 0096, 014 0101. 
Doctoral Theses: 012 0083. 
Documentation: 012 0073. 
Drug Abuse: 021 0116. 

Early Childhood Education: 012 0077, 021 0140, 021 0149. 
Early Reading: 021 0146. 
Economic Education: 021 0126. 
Economically Disadvantaged: 012 0019. 
Economics: 012 0039. 

Education: 012 0015, 012 0019, 012 0038, 012 0049, 012 0052, 012 0060, 

012 0051, 012 0086, 012 0087, 014 0105. 
Education Courses: 012 0064. 
Educational Accountability: 012 0016, 022 0210. 
Educational Administration: 012 0077, 012 0086. 

Educational Change: 012 0008, 012 0010, 012 0016, 012 0049, 012 0051, 
012 0052, 012 0053, 012 0054, 012 0055, 012 0056, 012 0068, 012 0074, 
012 0089, 021 0136, 022 0190, 022 0192, 022 0220, 027 0259, 027 0265, 
027 0277. 
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Educational 
Educational 

012 0047, 

012 

012 

022 

027 
Educational 
Educational 

021 QHS, 
Educational 
Educational 
Educational 
Educational 

012 0068, 

027 0259, 
Educational 

012 0076, 

021 0136 
027 0258. 

Educational 
Educational 
012 0054, 

022 0213, 
Educational 

027 0241 
Educational 
Educational 
Educational 

027 0249 
Educational 

012 0083, 

027 0246, 
Educational 
Educational 

012 0011 

012 

012 

012 

014 

021 

022 

027 

027 
Educational 
Educational 

012 0065, 

021 0123, 

021 0153, 

021 0163, 
Educational 
Educational 



Demand: 012 0089, 021 



0063, 

0072 

0088, 

0104, 

0141 

0168, 

0241 

0269, 



012 
012 
027 
027 



012 
014 
027 
027 



0138. 

011 0001, 012 0034, 
0063, 012 0064, 012 
0075, 012 0077, 012 
0097, 021 0141 , 021 
0243, 027 0246, 027 
0269. 
027 0277. 

012 0027, 012 0030, 
0149, 022 0168, 022 
0220. 

Games: 022 0172, 022 0198. 
History: 012 0038, 012 0039. 
Improvement: 012 0034, 012 0041, 
012 0072, 012 0074, 012 0077, 012 



Development: 
012 0056, 012 
0072, 
0089, 
0241 , 
0265, 
Environment: 
Experiments : 
021 0147, 021 
Finance: 022 



012 0042, 012 0043, 012 0044, 

0066, 012 0067, 012 0068, 

0078, 012 0079, 012 0084, 

0142, 022 0169, 022 0185, 

0255, 027 0256, 027 0259, 



012 0033, 012 0044, 012 0067, 
0205. 



012 0042, 012 0049, 012 0066, 
0079, 012 0089, 022 0192, 



Innovation: 012 0021, 012 0051, 012 0053, 012 0060, 012 0062, 
012 0077, 012 0088, 012 0089, 021 0119, 021 0126, 021 0131, 

021 0140, 021 0142, 021 0150, 021 0159, 022 0190, 022 0192, 

Needs: 012 0089, 022 0207. 

Objectives: 012 0011, 012 0012, 012 0040, 012 0045, 012 0046, 
012 0070, 021 0138, 022 0195, 022 0202, 022 0207, 022 0210, 

022 0215, 022 0218. 

Planning: 012 0042, 012 0051, 012 0086, 022 0190, 022 0220, 

Policy: 012 0045, 012 0084, 027 0241, 027 0245. 
Practice: 012 0052, 012 0076, 012 0088. 

Problems: 012 0063, 012 0066, 021 0138, 022 0190, 022 0214, 
027 0277. 

Programs: 012 0019, 012 0025, 012 0026. 012 0035, 012 0040, 
014 0096, 022 0185, 022 0191, 022 0210, 022 0221, 022 0222, 
027 0269, 036 0279. 
Psychology: 022 0175. 

Research: Oil 0001, Oil 0003, 012 0007, 012 0009, 012 0010, 
012 0012, 012 0014, 012 0039, 012 0041, 012 0047, 012 0058, 
012 0064, 012 0065, 012 0066, 012 0067, 012 0068, 012 0070, 
012 0074, 012 0075, 012 0077, 012 0079, 012 0083, 012 0084, 
012 0089, 014 0093, 014 0094, 014 0096, 014 0101 , 014 0103, 
014 0105, 021 0119, 021 0120, 021 0121 , 021 0131 , 021 0140, 

021 0142, 021 0158, 021 0159, 021 0161, 021 0162, 022 0167, 

022 0172, 022 0185, 022 0194, 022 0203, 022 0205, 022 0206, 
027 0243, 027 0^245, 027 0246, 027 0255, 027 0256, 027 0264, 
027 0277, 041 0281. 

Research Techniques: 012 0008. 

Researchers: 012 0010, 012 0017, 012.0018, 012 0019, 012 0023, 
012 0074, 012 0082, 014 0094, 014 0100, 014 0101 „ 014 0102, 
021 0143, 021 0144, 021 0148, 021 0150, 021 0151, 021 0152, 

021 0154, 021 0155, 021 0156, 021 0157, 021 0159, 021 0160, 

022 0185, 022 0194, 022 0206, 027 0269, 041 0281. 
Resources: 012 0012, 012 0056, 012 0066, 022 0190. 
Socijlogy: 027 0277. 
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Educational Specifications: 021 0147. 

Educational Strategies: 012 0021, 012 0044, 022 0190, 022 0192, 022 0208. 

Educational Technology: 012 0042, 012 0045, 012 0066. 012 0071. 027 0265. 

Educational Television: 012 0036, 021. 0142, 021 0161. 

Educational Testing: 022 0179. 

Educational Theories: 012 0074. 

Educationally Disadvantaged: 021 0149, 022 0222. 

Effective Teaching: 012 0046, 012 0058, 022 0210. 

Elementary School Curriculum: 012 0027. 

Elementary School Mathematics: 012 0085. 

Elementary School Teachers: 012 0058. 

Elementary Science: 012 0027. . 

Employment Qualifications: 021 0143, 021 0157. 

Engineering: 012 0069. 

English Education: 021 0151. 

English Instruction: 021 0112. 

Environmental Influences: 012 0086. 

Environmental Research: 012 0037. 

Estimated Costs: 012 0080. 

Evaluation: Oil 0003, 012 0008. 012 0009. 012 0015. 012 0016, 012 0017, 
012 0018, 012 0019, 012 0023, 012 0025. 012 0026, 012 0035, 012 0037. 
012 0040, 012 0042, 012 0043, 012 0044, 012 0051, 012 0064, 012 0081, 
012 0087, 012 0088, 012 0089, 021 0127, 021 0129. 021 0140, 021 0152. 

021 0162, 022 0169, 022 0179, 022 0203, 022 0224, 024 0227, 025 0233, 
027 0242, 027 0245, 027 0246. 027 0264, 027 0269, 027 0277, 036 0279, 
041 0281. 

Evaluation Criteria: Oil 0001, 012 0016, 012 0017, 012 0019, 012 0022, 
012 0025, 012 0036, 012 0043, 012 0080. 012 0085, 021 0155, 022 0178. 

022 0194, 022 0195, 022 0213, 022 0216, 022 0218, 022 0219, 022 0221 , 
025 0233. 

Evaluation Methods: 012 0018, 012 0026, 012 0028, 012 0029, 012 0030, 
012 0031 , 012 0032, 012 0033, 012 0035, 012 0037, 012 0040, 012 0080, 
012 0089, 021 0111, 021 0112. 021 0113, 021 0114, 021 0118, 021 0120, 

021 0161, 022 0175, 022 0177, 022 0178, 022 0188, 022 0194, 022 0212, 

022 0213, 022 0214. 022 0218, 022 0221, 022 0223, 022 0224, 023 0225, 
024 0227, 027 0242. 

Evaluation Methods and Models: 022 0195. 

Evaluation Needs: 012 0016, 012 0021, 012 0036, 027 0242. 

Evaluation Techniques: 012 0016, 012 0029, 012 0088, 022 0168, 022 0175. 

022 0177, 022 0178, 022 0188, 022 0194, 022 0204, 022 0213, 022 0214, 

022 0216, 022 0221, 022 0223, 022 0224, 023 0225. 
Evaluative Techniques: 022 0172. 
Evaluative Thinking: 022 0172, 022 0214, 022 0218. 
Experience: 012 0062. 
Experiment Stations: 012 0060. 
Experimental Curriculum: 012 0034, 022 0167. 
Experimental Groups: 012 0033. 

Experimental Programs: 012 0078, 014 0094, 014 0097, 021 0146, 021 0147, 

021 0149. 
Experiments: 012 0009. 
Expository Writing: 022 0186. 
Failure Factors: 022 0200, 022 0214. 
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Federal Aid: 012 0004, 012 0005, 012 0006, 012 0007, 012 0013, 012 0024, 

012 0071 , 012 0072 , 012 0074 , 012 0077 , 012 0079 , 014 0104. 
Federal Programs: 012 0019, 012 0077. 

Feedback: 012 0020, 012 0021, 012 0048, 012 0061, 012 0085, 022 0175, 
022 0217. 

Field Experience: 022 0169, 022 0203, 022 0208, 027 0241, 027 0243, 

027 0245, 027 0249, 027 0264, 029 0265. 
Field Interviews: 022 0189. 

Field Studies: 012 0010, 012 0044, 012 0064, 012 0085. 
Film Production: 021 0155, 021 0161. 
Films: 022 0189. 
Filmstrips: 022 0179. 
Financial Needs: 014 0104. 

Financial Support: 014 0095, 014 0101, 021 0161. 
Fixed Response: 022 0212. 
Fixed Sequence: 022 0175. 
Flexible Scheduling: 021 0119. 
Flow Charts: 012 0043. 

Formative Evaluation: 012 0043, 012 0044, 021 0111, 021 0112, 021 0115. 

021 0118, 021 0131, 021 0138, 021 0158, 022 0169, 022 0172, 022 0178, 

022 0187, 022 0208, 022 0214, 022 0221, 022 0223. 
Game Theory: 022 0198. 

Goal Orientation: 012 0018 , 022 0207. 

Grades 11-12: 021 0128. 

Grades 9 - 12: 021 0123. 

Grades 1 - 8: 021 0121 , 021 0129. 

Grades 1 - 6: 021 0114, 021 0118, 021 0125, 021 0127. 

Grades 1-3: 021 0115, 021 0124. 

Grades 1 - 12: 021 0126. 

Grades 7-9: 021 0113. 

Grading: 012 0068. 

Graduate Professors: 014 0105. 

Graduate Study: 012 0047, 014 0094, 014 0101, 014 0105. 

Graduate Surveys: 014 0096, 014 01 OG. 

Graphs: 012 0026. 

Group Status: 012 0048. 

Groups: 012 0031 . 

Guidelines: 012 0050. 

Guides: Oil 0001, 012 0016. 

Health Education: 021 0115. 

High School Students: 021 0120. 

Higher Education: 014 0105. 

Historical Reviews: 014 0095. 

History: 012 0006. 

Home Visits: 021 0149. 

Improvement Programs: 012 0096. 

Incentive Systems: 022 0197. 

Indexes (Locators): 014 0093, 021 0160. 

Individual Characteristics: 012 0026, 014 0101. 

Individual Instruction: 012 0018, 012 0077, 012 0085, 021 0127, 021 0163, 

022 0169, 022 0187, 022 0192, 022 0207, 027 0249, 027 0265. 
Individualized Curriculum: 014 0113. 
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Industrial Personnel : 036 0279. 
Industrial Technology: 012 0006. 
Industry: 012 0040. 

Information Centers: 012 0049. 012 0056, 012 0063, 021 0142, 021 0160, 
022 0176. 

Information Dissemination: 012 0029, 012 0032, 012 0044, 012 0050, 012 0063, 
012 0079, 012 0084, 014 0105, 021 0142, 021 0158, 021 0160, 022 0170, 
025 0233. 

Information Needs: 012 0021 , 012 0066, 012 0070, 012 0081 , 021 0136, 

022 0170, 022 0183. 
Information Networks: 021 0136. 

Information Retrieval: 021 0142, 022 0169, 022 0170, 022 0183. 
Information Science: 012 0063, 021 0160. 
Information Seeking: 022 0170, 022 0183. 
Information Services: 012 0049, 012 0054, 012 0061. 
Information Sources: 012 0049, 022 0176, 022 0183. 
Information Storage: 012 0032. 

Information Systems: 012 0032, 012 0048, 012 0049, 012 0088, 021 0157, 

022 0176. 
Information Theory: 012 0048. 

Information Utilization: 012 0021, 012 0050, 012 0054, 012 0059. 
Innovation: 012 0050, 012 0051, 012 0057, 012 0060, 012 0061, 012 0067, 

012 0073. 
Input Output: 012 0037, 012 0068. 
Input Output Devices: 021 0121. 
Inquiry Training: 021 0123, 021 0130. 
Inservice Education: 014 0100. 

Inservice Teacher Education: 021 0112, 021 0114, 021 0126, 021 0128, 

021 0130. 

Institutes (Training Programs): 022 0177, 022 0191, 022 0203, 027 0245, 

027 0249, 027 0255, 027 0258, 027 0264. 
Institutional Environment: 027 0277. 

Institutional Research: 021 0147, 022 0167, 027 0255, 027 0256. 
Institutions: 014 0101. 
Instructional Aids: 022 0219. 

Instructional Design: 012 0025, 012 0041, 022 0187, 025 0233. 
Instructional Improvement: 022 0210. 
Instructional Innovation: 012 0030. 
Instructional Material Evaluation: 021 0132. 

Instructional Materials: 012 0010, 012 0022, 012 0041, 012 0045, 012 0063, 
012 0064, 021 0125, 021 0138, 021 0157, 021 0163, 022 0168, 022 0169, 

022 0170, 022 0179, 022 0187, 022 0189, 022 0192, 022 0205, 022 0207, 
022 0219, 025 0233, 027 0249. 

Instructional Media: 021 0141, 021 0163. 
Instructional Programs: 012 0018, 022 0187. 
Instructional Systems: 012 0046, 021 0112. 
Instructional Technology: 021 0141, 022 0217, 024 0227. 
Instructional Television: 012 0036. 
Integrated Activities: 012 0054. 
Interaction Process Analysis: 012 0017. 
Interdisciplinary Approach: 012 0053, 012 0054. 
Interpersonal Relationship: 012 0048. 
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Intervention: 012 0075. 

Interviews: 014 0096, 021 0162, 022 0189. 

Investigations: 012 0014. 

Item Analysis: 012 0028, 022 0177, 022 0199. 

Item Sampling: 022 0177. 

Job Analysis: 012 0046. 

Junior College Libraries: 021 0147. 

Junior Colleges: 022 0210. 

Kindergarten: 012 0028, 021 0111, 021 0114, 021 0115, 021 0118, 021 0124, 

021 0125, 021 0126, 021 0127. 
Knowledge Level: 012 0066. 
Laboratories: 012 0072, 012 0085. 
Laboratory Training: 022 0191. 
Language Arts: 021 0124, 021 0159. 
Language Development: 021 0131. 
Leadership: 012 0062, 022 0192. 
Leadership Styles: 012 0076. 
Leadership Training: 012 0020, 021 0141. 
Learning: 012 0027, 012 0067, 012 0079. 
Learning Activities: 021 0140. 
Learning Experience: 012 0012, 012 0045. 
Learning Modalities: 021 0125, 021 0156, 024 0227. 
Learning Motivation: 022 0197. 

Learning Processes: 012 0012, 014 0104. 
Lecture: • 012 0036. 

Level One - HS (Grades 1 - 12): 021 0132. 
Library Research: 021 0147. 
Library Technical Processes: 021 0157. 
Literature Guides: 022 0176. 

Literature Reviews: Oil 0003, 012 0031, 012 0053, 012 0089, 012 0101, 

022 0203. 

Long Achievement Factors: 021 0145. 

Longitudinal Studies: 021 0145. 

Low Achievement Factors: 022 0200, 022 0214. 

Low Achieves: 021 0145. 

Magnetic Tape Cassettes: 021 0163, 024 0227. 

Management: 012 0013, 012 0070, 012 0079, 021 0158, 021 0159, 021 0160, 

021 0161. 

Management Education: 027 0241, 027 0256. 
Management Systems: 012 0083. 

Manpower Development: Oil 0003, 014 0094, 014 0101, 027 0259. 
Manpower Needs: Oil 0003, 012 0070, 027 0259, 036 0279. 
Manpower Utilization: 012 0082. 
Manuals: 022 0190. 

Marketing: 012 0040, 012 0084, 021 0138, 022 0169, 025 0233. 

Mass Media: 012 0049. 

Master Tapes (Audio): 024 0227. 

Material Development: 021 0111, 021 0115, 021 0122, 021 0124, 022 0187, 

022 0188, 023 >225, 025 0233. 

Mathematic Instruction: 021 0121, 021 0124, 021 0127, 021 0131. 
Mathematical Applications: 022 0193. 
Mathematical Enrichment: 022 0193. 
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Mathematics Materials: 012 0085. 
Measurement: 012 0009, 012 0026, 022 0179. 

Measurement Goals: Oil 0003, 012 0017, 021 0140, 021 0141, 021 0142, 

021 0151, 021 0152, 022 0202, 022 0213, 022 0215, 022 0218. 
Measurement Instruments: 012 0029 , 012 0031, 012 0032 , 012 0038,. 022 01 79. 
Measurement Techniques: 021 0120, 022 0177, 022 0188, 022 0194, 022 0199, 

022 0213, 022 0223, 023 0225. 
Meetings: 012 0055. 

Mental Health: 012 0061. 

Methodology: 012 0034, 012 0038, 012 0040, 012 0042, 012 0081. 
Methods: 012 0038. 

Methods Research: 012 0038, 012 0059. 
Mexican Americans: 012 0087. 
Microforms: G21 0140, 021 0147. 
Microteaching: 012 0064, 021 0131. 
Military Air Facilities: 012 0087. 
Military Organizations: 012 0083. 
Milwaukee: 012 0008. 
Minimally Brain Injured: 021 0117. 
Minority Group Children: 012 0087. 

Minority Role: 022 0203, 027 0243, 027 0255, 027 0256, 027 0264, 027 0277. 
Mobile Classrooms: 021 0149. 

Models: 012 0008, 012 0010, 012 0011, 012 0012, 012 0015, 012 0016, Oi2 0017, 
012 0019, 012 0020, 012 0021, 012 0023, 012 0024, 012 0025, 012 0032, 
012 0035, 012 0037, 012 0043, 012 0044, 012 0045, 012 0052, 012 0053, 
012 0055, 012 0061, 012 0063, 012 0068, 012 0069, 012 0083, 012 0084. 

Morale: 021 0151. 

Motivation: 012 0048. 022 0197. 

Motivation Techniques: 012 0022, 022 0197. 

Multimedia Instruction: 021 0112, 021 0122, 021 0156. 

Museums: 021 0125. 

Music Education: 021 0159. 

National Programs: 012 0005, 012 0013. 

National Surveys: 012 0071. 

Natural Sciences: 014 0095. 

Needs: 022 0207. 

Networks: 012 0082, 012 0083. 

New York City: 014 0101 , 014 0105. 

Norms: 021 0117. 

Nursery Schools: 012 0080. 

Objectives: Oil 0003, 012 0007, 012 0015, 012 0017, 012 0018, 012 0022, 
012 0040, 012 0065, 012 0075, 012 0082, 014 0101, 021 0140, 021 0141, 
021 0142, 022 0215. 

Observation: 012 0015, 022 0177. 

Occupational Clusters: 021 0128. 

Occupational Guidance: 021 0128, 021 0154. 

On-Line Systems: 022 0172, 022 0183. 

Cn-the-Job Training: 027 0258. 

Operant Conditioning: 021 0115. 

Opinions: 012 0015. 

Organization: 012 0051, 012 0073, 014 0095, 014 0105, 021 0140. 
Organizational Change: 012 0059, 021 0T36. 



ERIC 



Participant Involvement: 012 0055. 
Participation: 022 0208. 
Peer Teaching: 021 0112, 021 0127. 
Perception: 012 0011, 012 0027. 
Perception Tests: 021 0117. 
Perceptual Development: 021 0117. 
Perceptual Motor Learning: 021 0117. 
Perceptually Handicapped: 021 0117. 
Performance: 012 0070, 022 0200. 

Performance Criteria: 012 0043, 022 0178, 022 0202, 022 0213, 022 0214, 

022 0216, 022 0218, 022 0224. 
Performance Factors: 022 0200, 022 0214. 
Performance Specifications: 022 0213, 022 0218. 
Performance Tests: 022 0178, 022 0213, 022 0223. 
Personnel : 014 0093. 
Personnel Evaluation: 022 0200. 
Personnel Management: 021 0161, 021 0162. 

Personnel Needs: 012 0078, 014 0097, 022 0185, 027 0269, 036 0279. 

Philosophy: 022 0205. 

Phonics: 021 0111. 

Physical Education: 012 0058. 

Physical Sciences: 012 0061. 

Physics: 012 0051. 

Pilot Projects: 012 0055, 022 0197. 

Plannina: 012 0052, 027 0241. • 

Policy: 012 0084, 014 0095, 027 0241, 027 0245. 

Policy Formation: 027 0241. 

Political Influences: 012 0030. 

Portland: 012 0038. 

Positive Reinforcement: 021 0111. 

Postgraduate Education: 014 0096. 

Poverty Programs: 012 0025. 

Prediction: 022 0205. 

Predictive Validity: 022 0199. 

Preschool Children: 012 0080. 

Preschool Education: 021 0120, 021 0161. 

Preschool Programs: 012 0029, 021 0124. 

Probability: 012 0004. 

Problem Solving: Oil 0003, 012 0050, 012 0089, 022 0170. 
Problems: 012 0005, 012 0075, 012 0086. 
Production Techniques: 012 0036. 
Productivity: 012 0034, 012 0068. 

Professional Continuing Education: 021 0163, 02? 0169, 022 0192. 
Professional Education: 014 0105, 021 0123, 021 0138, 021 0143, 021 0144, 
021 0148, 021 0150, 021 0151, 021 0152, 021 0153, 021 0154, 021 0155, 

021 0157, 021 0160. 
Professional Personnel: 012 0069. 

Professional Training: 012 0047, 012 0079, 014 0094, 014 0101, 022 0169, 

022 0188, 022 0192, 022 0203, 022 0206, 022 0207, 022 0208, 022 0210, 

023 0225, 027 0241, 027 0243, 027 0245, 027 0249, 027 0255, 027 0256, 
027 0264, 027 0265, 027 0277. 
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Program Administration: 012 0070, 021 0116, 021 0118, 021 0123, 021 0143, 
021 0150, 021 0151, 021 0152, 021 0153, 021 0154, 021 0155, 021 0156, 

021 0157, 022 0220. 

Program Coordination: 012 0082. 

Program Description: 021 0145, 021 0147, 022 0169, 022 0192, 022 0203, 

022 0207, 022 0208, 027 0241, 027 0243, 027 0245, 027 0249, 027 0255, 
027 0256, 027 0264, 027 0265, 027 0277. 

Program Development: 012 0020, 012 0025, 012 0041, 027 0255, 027 0277. 
Program Design: 012 0021, 012 0036, 012 0040, 022 0168, 022 0192, 022 0208. 
Program Effectiveness: 012 0021, 012 0027, 012 0037, 012 0216, 022 0224. 
Program Evaluation: Oil 0001, 012 0016, 012 0021, 012 0029, 012 0037, 
012 0065, 012 0082, 012 0083, 012 0085, 014 0096, 021 0140, 021 0141, 

021 0142, 022 0178, 022 0192, 022 0195, 022 0213, 022 0218, 022 0221, 

022 0223, 022 0224, 027 0277. 
Program Improvement: 022 0208, 027 0255. 
Program Length: 012 0036. 

Program Planning: 012 0021, 012 0024, 012 0082, 022 0169, 022 0208, 
022 0220. 

Program Proposals: 012 0024, 022 0195, 022 0220, 022 0221, 027 0245, 
027 0255. 

Programmed Instruction: 012 0007, 021 0121, 021 0156, 022 0174, 022 0175, 

022 0199, 022 0202, 022 0212, 022 0217. 
Programmed Materials: 021 0115, 022 0174, 022 0212, 022 0217. 
Programmed Tutoring: 021 0122. 
Programmers: 022 0175, 022 0212, 022 0217. 
Programning; 021 0119, 021 0121, 022 0175, 022 0212, 022 0217. 
Programming (Broadcast): 021 0120. 
Programming Methods: 022 0175, 022 0212, 022 0217. 
Programming Problems: 022 0217. 
Project Applications: 036 0279, 041 0281. 
Project Training Methods: 012 0085, 012 0087. 
Projects: 012 0023, 012 0024, 012 0030, 012 0031, 012 0035. 
Prompting: 022 0217. 

Protocol Materials: 021 0129, 021 0130, 021 0155. 

Psychoeducational Processes: 012 0011. 

Psych ometrics: 012 0038. 

Psychomotor Objectives: 022 0195, 022 0215. 

Public Education: 012 0034. 

Public Policy: 012 0005, 012 0006, 012 0013, 012 0077. 

Public Relations: 021 0161. 

Public Schools: 012 0023. 

qualifications: 021 0123, 021 0130, 021 0158. 

Question Answer Interviews: 022 0189. 

Questionnaires: 012 0022, 014 0093, 014 0096, 014 0104, 021 0144, 021 0148, 

021 0150, 022 0179, 022 0188, 022 0189, 023 0225. 
Rating Scales: 012 0022. 
Reading Development: 021 0146. 
Reading Improvement: 021 0146, 022 0222. 

Reading Instruction: 012 0025, 021 0111, 021 0115, 021 0122, 021 0124, 

021 0162. 
Reading Materials: 021 0111. 
Reading Processes: 921 0146. 



ERIC 



Reading Programs : 021 0142. 
Reading Research: 021 0140, 021 0146. 
Reading Skills: 012 0087, 021 0146. 
Reference Books: 014 0093, 022 0176. 
Reference Materials: 022 0183. 

Regional Laboratories: 012 0065, 012 0076, 012 0088. 021 0111, 021 0127, 

021 0153, 021 0158, 036 0279. 
Relevance (Education): 012 0071. 
Reliability: 012 0009. 
Remedial Arithmetic: 021 0121. 
Remedial Instruction: 012 0010. 
Reports: 012 0033, 021 0132, 022 0186. 

Research: 012 0004, 012 0008. 012 0013, 012 0042, 012 0048, 012 0050, 

012 0058. 012 0060, 012 0081 , 014 0095, 014 0105. 
Research Apprenticeships: 014 0100, 014 0101, 014 0102, 014 0104, 

021 0144, 022 0206. 

Research Committees: 012 0010, 012 0015. 
Research Coordinating Units: 021 0136. 

Research Criteria: 012 0011, 021 0145, 021 0146, 021 0147, 021 0149, 

022 0194, 022 0205. 

Research Design: 012 0011 , 012 0014, 012 0016, 012 0026, 012 0031 , 
012 0035, 012 0069, 021 0145, 021 0146, 021 0147, 021 0148, 021 0149, 

021 0156, 022 0167, 022 0168, 022 0185, 022 0188, 022 0194, 022 0203, 

022 0205, 023 0225, 027 0249, 027 0264, 027 0269. 

Research and Development Centers: Oil 0003, 012 0047, 012 0059, 012 0065, 
012 0067, 012 0072, 012 0074, 012 0077, 012 0078, 012 0084, 012 0085, 
012 0088, 014 0097, 014 0101, 022 0224, 027 0255, 027 0259, 036 0279. 

Research Directors: 012 0078, 014 0097. 
' Research Methodology: Oil 0001, 012 0008, 012 0014, 012 0029, 012 0030, 
012 0033. 012 0039, 012 0069, 012 0080, 012 0083, 012 0089, 021 0103, 

022 0167, 022 0168, 022 0188, 022 0189, 022 0194, 022 0203, 022 0205, 

023 0225, 027 0245. 

Research Needs: 012 0067, 014 0100, 036 0279. 
Research Opportunities: 014 0104. 

Research Problems: 012 0016, 012 0033, 012 0035, 012 0040, 012 0059, 
012 0078, 012 0081, 014 0097, 014 0104, 021 0144, 021 0156, 022 0203, 
027 0249. 

Research Projects: 012 0078, 012 0082. 014 0097, 021 0143, 021 0144, 
021 0145, 021 0146, 021 0147, 021 0149, 021 0150, 021 0151 , 021 0152, 

021 0153, 021 0154, 021 0155, 021 0156, 021 0157, 022 0167, 022 0168, 

022 0194, 027 0249, 027 0264. 

Research Proposals: 012 0075, 022 0168, 022 0194, 022 0203. 
Research Reviews (Publications): 012 0076. 

Research Skills: 014 0093. 014 0102. 021 0143, 021 0144, 021 0148, 
021 0150, 021 0151, 021 0152, 021 0153, 021 0154, 021 0155. 021 0156, 

021 0157, 021 0163, 022 0167, 022 0168, 022 0172, 022 0179, 022 0189, 

022 0194, 022 0203, 022 0205, 022 0206, 027 0249, 027 0255, 027 0256, 
027 0265, 027 0277. 

Research Tools: 022 0168, 022 0194. 

Research Utilization: Oil 0001, Oil 0003, 012 0007, 012 0048, 012 0050, 
012 0052, 012 0059, 012 0061, 012 0074, 012 0076, 012 0078, 012 0080, 
012 0081, 012 0084, 012 0088, 012 0089, 014 0097, 021 0142, 027 0249. 
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Researchers: 012 0058. 012 0059, 012 0078, 012 0088, 014 0093, 014 0097. 

Resource Allocations: 012 0082. 

Resource Guides: 022 0190. 

Risk: 012 0010, 012 0020. 

Role Playing: 022 0170, 022 0189. 

Role Theory: 014 0095. 

Russian: 012 0067. 

Sampling: 012 0032, 012 0038, 021 0150, 022 0193, 022 0204. 

Schedule Modules: 021 0119. 

School Activities: 012 0068. 

School Funds: 012 0024. 

School Improvement: 012 0068. 

School Involvement: 012 0020. 

School Personnel: 022 0192, 022 0203, 022 0207, 027 0243, 027 0249, 

027 0265. 
School Statistics: 021 0148. 
School Systems: 012 0049, 012 0051, 012 0060. 
School Visitation: Oil 0001, 012 0076, 021 0142. 
Schools of Education: 012 0047. 
Science Course Improvement Project: 012 0011. 
Science Curriculum: 012 0031 , 012 0064. 
Science Education: 012 0005, 012 0011, 012 0012, 012 0056. 
Science History: 014 0095. 
Science Institutes: 012 0071. 

Science Instruction: 012 0028, 021 0113, 021 0114, 021 0118, 021 0130. 

Science Laboratories: 012 0006, 014 0095. 

Science Materials: 012 0027, 021 0113, 021 0114, 021 0118. 

Scientific Concepts: 012 0075. 

Scientific Enterprise: 014 0095. 

Scientific Manpower: 012 0013. 

Scientific Methodology: 012 0011, 012 0052. 012 0069, 012 0075. 
Scientific Research: 012 0004, 012 0005, 012 0006, 012 0007, 012 0013, 

012 0050, 012 0052, 014 0095. 
Scientists: 012 0004, 012 0006, 014 0095. 
Scripts: 021 0161. 
Search Strategies: 022 0183. 
Secondary School Mathematics: 022 0193. 
Secondary School Teachers: 012 0058. 
Selection: 022 0219. 
Self Concept: 012 0026. 
Seminars: 022 0207, 027 0241, 027 0249. 
Sensory Training: 021 0117. 
Sequential Approach: 012 0046. 
Sequential Learning: 021 0121. 
Sequential Programs: 022 0175. 
Services: 012 0047. 
Short Courses: 012 0054. 

Simulation: 022 0170, 022 0172, 022 0185, 022 0187, 022 0198, 022 0207, 

027 0258. 
Skills Development: 021 0128. 
Social Change: 012 0035, 014 0095, 022 0192. 
Social Class: 012 0011 . 
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Social Factors: 012 0050, 012 0070, 021 0144. 

Social Influences: 012 0006. 

Social Problems: 012 0035, 012 0071. 

Social Sciences: 012 0013, 012 0039, 012 0052, 012 0061, 012 0071. 
Social Services: 012 0071. 

Social Studies: 012 0005, 012 0020, 021 0123, 021 0125, 021 0129, 

021 0159. 

Social Values: 014 0095. 

Socially Disadvantaged: 021 0149. 

Sociology: 014 0095. 

Spanish Americans: 012 0080. 

Special Education: 012 0122. 

Specialists: 027 0259, 041 0281. 

Speeches: 012 0065. 

Staff Orientation: 021 0119. 

Staff Role: Oil 0003. 

Staff Utilization: 036 0279. 

Standard Error of Measurement: 022 0199. 

Standardized Tests: 022 0177. 

Standards: 012 0015, 012 0018. 

State Action: 012 0062. 

State Colleges: 012 0058. 

State Departments of Education: 012 0058, 021 0136, 022 0220. 

State Federal- Aid: 022 0221 , 022 0222. 

State Federal Support: 022 0220. 

State Programs: 022 0220, 022 0222. 

State Surveys: 022 0222. 

State Universities: 012 0058. 

Statistical Analysis: 012 0009, 012 0080, 014 0101, 022 0174, 022 0188, 

022 0193, 022 0194, 022 0204, 023 0225. 
Statistical Bias: 022 0193, 022 0204. 
Statistical Studies: 021 0145. 
Statistical Surveys: 022 0222. 

Statistics: 012 0004, 022 0174, 022 0193, 022 0204. 
Status: 012 0048, 022 0205. 
Student Attitudes: 012 0027. 
Student Development: 012 0012. 

Student Evaluation: 012 0012, 012 0017, 012 0018, 012 0020, 022 0214. 
Student Motivation: 022 0197. 
Subject Index Terms: 022 0183. 

Summative Evaluation: Oil 0003, 012 0043, 012 0044, 012 0046, 021 0163, 

022 0178, 022 0195, 022 0221, 022 0224, 025 0233, 027 0265. 
Summer Schools: 012 0058. 
Supervisory Training: 022 0191. 
Supplementary Educational Centers: 022 0220. 
Supplementary Textbooks: 022 0193, 022 0199. 
Surveys: 012 0038, 014 0104, 014 0105, 022 0189. 
Symposia: 012 0019. 

Systems Analysis: 012 0037, 012 0086, 012 0087. 
Systems Approach: 012 0037, 012 0C46, 012 0048, 012 0054, 012 0077, 
012 0086, 012 0087, 021 0113, 022 0198. 
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Systems Concepts: 022 0198. 

Systems Development: 012 0054, 012 0069. 012 0072, 012 0087^ 021 0158, 

022 0198, 022 0208. 
Tape Recordings: 021 0163, 022 0207, 027 0249. 
Task Analysis: 012 0082, 022 0187, 022 0200. 
Task Performance: 022 0200. 
Taxonomy: 012 0070. 
Teacher Aids: 021 0127. 
Teacher Behavior: 021 0111. 

Teacher Developed Materials: 021 0113, 021 0126, 021 0129. 

Teacher Education: 012 0043, 012 0064, 012 0072, 012 0077, 012 0104, 

021 0117, 021 0131 , 021 0141 , 021 0159. 
Teacher Education Curriculum: 012 0046. 
Teacher Motivation: 022 0197. 

Teacher Programs: 012 0046, 021 0114, 027 0259. 

Teacher Role: 012 0046, 021 0114, 021 0130. 

Teacher Workshops: 012 0058. 

Teachers Colleges: 014 0104. 

Teaching: 012 0012. 

Teaching Machines: 021 0115, 021 0122. 

Teaching Methods: 012 0041. 

Teaching Models: 012 0064, 02? 0191. 

Teaching Skills: 021 0131. 

Teaching Techniques: 012 0036, 022 0185. 

Team Teaching: 021 0143. 

Team Training: 021 0143, 027 0265. 

Teamwork: 021 0150. 

Technical Reports: 022 0169, 022 0186, 022 0203. 

Technical Writing: 021 0160, 021 0162. 022 0186, 022 0194. 

Technological Advancement: 012 0006. 

Technology: 012 0005, 012 0006, 012 0013, 012 0048. 

Telephone Communication Systems: 012 0048. 

Televised Instruction: 021 0149. 

Television: 021 0120. 

Test Construction: 021 0117, 021 0123, 021 0129, 021 0148, 022 0177, 

022 0179, 022 0187, 022 0199. 

Test Interpretation: 022 0179, 022 0199, 022 0216. 

Test Reliability: 022 0179, 022 0199. 

Test Results: 022 0216. 

Test Reviews: 022 0199. 

Test Selection: 022 0177, 022 0179. 

Test Validity: 022 0199. 

Testing Problems: 012 0029. 

Tests: 021 0129, 022 0223. 

Theoretical Criticism:^ 022 0205. 

Theories: 012 0039, 012 0075. 

Thesauri: 022 0183. 

Time: 012 0081 . 

Title III Projects: 012 0088. 

Trainees: 022 0206. 

Trainers: 022 0191. 

Training: 012 0061, 022 0185. 022 0191. 022 0206. 027 0259. 
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Training Objectives: 022 0187,. 022 0191 , 022 0202, 022 0206, 022 0210, 

022 0215, 027 0269. 
Training Techniques: 012 0046, 012 0078, 012 0087, 014 0097, 021 0163, 

022 0170, 022 0172, 022 0179, 022 0187, 022 0189, 022 0191, 022 0205, 

022 0206, 022 0210. 
Transfer of Training: 014 0100, 014 0102. 
Transparancies: 022 0207, 022 0208. 
True Scores: 022 0199. 
Tutoring: 021 0120, 021 0131. 
Typewriting: 012 0122. 
Undergraduate Study: 014 0094. 
Unemployed: 014 0100. 

Units of Study (Subject Fields): 012 0027, 012 0028. 

Universities: 012 0005, 012 0006, 012 0047, 012 0071, 012 0074, 014 0095, 
014 0105. 

Urban Education: 012 0023, 022 0223, 027 0277. 

Urban Schools: 012 0033. 

Use Studies: 021 0138, 021 0147. 

Validity: 012 0009. 

Values: 012 0017, 012 0019, 012 0024, 012 0048, 012 0057. 

Videotape Recordings: 021 0131. 027 0249. 

Viewing Time: 012 0036. 

Vocational Agriculture Teachers: 012 0062. 

Vocational Development: 021 0141. 

Vocational Education: 021 0128, 021 0154. 

Volunteer Training: 021 0120. 

Western Civilization: 014 0095. 

Work Attitudes: 022 0200. 

Work Environment: 022 0200. 

Work Experience: 021 0153. 

Work Experience Programs: 014 0100, 014 0102. 

Work Simplifications: 012 0082, 022 0200. 

Writing: 012 0033. 

Writing Skills: 021 0158, 022 0186. 

Written Language: 012 0055. 
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